# How to Use Demucs for Audio Source Separation in the ailia‑models Repository

> Learn how to use Demucs for audio source separation in ailia-models. Separate stereo audio into drums, bass, other, and vocals with efficient chunk-based inference and ONNX Runtime support.

- Repository: [axinc-ai/ailia-models](https://github.com/axinc-ai/ailia-models)
- Tags: tutorial
- Published: 2026-02-26

---

**The Demucs implementation in ailia‑models separates stereo audio into four stems—drums, bass, other, and vocals—using chunk‑based inference with optional ONNX Runtime support.**

The **ailia‑models** repository by axinc‑ai provides a ready‑to‑use pipeline for music source separation based on Facebook Research’s Demucs architecture. This guide explains how to run inference using the `htdemucs_ft` model, configure runtime backends, and process audio programmatically or via CLI.

## Model Architecture and Weights

The Demucs implementation defines a `htdemucs_ft` model composed of four distinct ONNX networks—one for each target source (drums, bass, other, vocals)—and a segment length expressed as a `Fraction` in the initialization code.

In [`audio_processing/demucs/demucs.py`](https://github.com/axinc-ai/ailia-models/blob/main/audio_processing/demucs/demucs.py), the `main()` function handles model setup between lines 45–63. It first invokes `check_and_download_models` (imported from [`util/model_utils.py`](https://github.com/axinc-ai/ailia-models/blob/main/util/model_utils.py)) to fetch the `.onnx` and `.prototxt` files from a Google Storage bucket on first run (lines 65–70). Internet access is required only for this initial download.

## Runtime Selection: Ailia SDK vs. ONNX Runtime

You can choose between two inference backends via the `--onnx` CLI flag:

- **Ailia SDK** (default): When `--onnx` is omitted, the script loads each source network using `ailia.Net` with an `env_id` derived from CLI arguments (lines 73–94).
- **ONNX Runtime**: When `--onnx` is supplied, the script creates an `onnxruntime.InferenceSession` for each of the four source models, removing the dependency on the ailia SDK.

Use the ONNX Runtime option for environments where the ailia SDK is not installed or when debugging model behavior in pure ONNX tools.

## Audio Loading and Preprocessing

The `load_audio` function (lines 81–103) reads input files using **librosa**, with an optional **ffmpeg** fallback if enabled. It automatically resamples audio to 44.1 kHz and ensures stereo output by duplicating mono channels, guaranteeing a tensor shape of `(2, T)` where `T` is the number of samples. This normalization step is critical because the Demucs model expects standard CD‑quality stereo input.

## Chunk‑Based Inference Pipeline

The core separation logic resides in `apply_model` (lines 124–152), which processes long audio files in overlapping chunks to manage memory usage and maintain temporal coherence.

Key parameters controlling this behavior include:

- **segment**: Length of each analysis window.
- **overlap**: Fractional overlap between consecutive chunks.
- **transition_power**: Exponent used for the weighted triangular window that blends chunk outputs.
- **shifts**: Optional random temporal offsets applied to the input; results are averaged across shifts to reduce boundary artifacts.

When `split=True`, the algorithm iterates over the mixture using the weighted window to smoothly merge predictions from adjacent segments.

## Source Isolation and Output Generation

Inside the `predict` function (lines 75–99), the pipeline isolates each source by temporarily swapping the target network into `models["net"]`, running `apply_model`, and zeroing out the outputs for all other sources before accumulation. After processing all four networks, the final estimates are denormalized to the original audio scale to preserve input loudness levels.

Results are saved using either:
- **MP3**: If the optional *lameenc* encoder is available.
- **WAV**: Fallback to *soundfile* for uncompressed PCM output.

## Command‑Line Usage

The script exposes standard arguments through `get_base_parser` and `update_parser` (from [`util/arg_utils.py`](https://github.com/axinc-ai/ailia-models/blob/main/util/arg_utils.py)), including `--input`, `--savepath`, `--model_type`, and `--onnx`.

Separate the default test audio into four stems:

```bash
python3 audio_processing/demucs/demucs.py

```

Process a custom file with ONNX Runtime and specify an output directory:

```bash
python3 audio_processing/demucs/demucs.py \
  --input path/to/song.wav \
  --savepath results/ \
  --onnx

```

## Programmatic Integration

You can invoke the pipeline from Python by importing the main entry point:

```python
from audio_processing.demucs.demucs import main as demucs_main

# Executes with sys.argv; handles all CLI arguments internally

demucs_main()

```

For advanced use cases involving NumPy arrays directly, you would replicate the initialization steps from `main()` to populate the `models` dictionary, then call `predict(models, audio_array)`. Note that `load_audio` returns a NumPy array of shape `(2, T)` ready for inference.

## Summary

- The Demucs model in [`audio_processing/demucs/demucs.py`](https://github.com/axinc-ai/ailia-models/blob/main/audio_processing/demucs/demucs.py) separates stereo audio into drums, bass, other, and vocals using four independent ONNX networks.
- Automatic weight downloads via `check_and_download_models` require internet only on first run.
- Use the `--onnx` flag to switch from the Ailia SDK to ONNX Runtime for inference.
- Audio is resampled to 44.1 kHz and processed in overlapping chunks with optional shifts to improve quality.
- Output files are written as MP3 (with lameenc) or WAV (with soundfile) based on encoder availability.

## Frequently Asked Questions

### What audio file formats are supported as input?

The `load_audio` function relies on **librosa**, which supports WAV, MP3, FLAC, and OGG. If **ffmpeg** is installed and enabled, the function can also decode formats outside librosa’s native support, provided they can be resampled to 44.1 kHz stereo.

### Can I run the model without installing the ailia SDK?

Yes. Supply the `--onnx` flag when executing [`demucs.py`](https://github.com/axinc-ai/ailia-models/blob/main/demucs.py). This forces the script to use `onnxruntime.InferenceSession` instead of `ailia.Net`, allowing the pipeline to run on systems where only ONNX Runtime is available.

### How does the chunk‑based processing handle long audio files?

The `apply_model` function divides the input into segments defined by the `segment` parameter, processes each chunk separately, and blends overlapping regions using a weighted triangular window controlled by `transition_power`. Optional shifts average multiple temporal offsets to minimize audible artifacts at chunk boundaries.

### Why are there four separate ONNX models instead of one?

The `htdemucs_ft` architecture uses source‑specific networks to isolate drums, bass, other, and vocals independently. During `predict`, each network is loaded sequentially as `models["net"]`, and the outputs for non‑target sources are masked to zero before accumulation, enabling precise stem extraction without cross‑source interference.