How Deep-Live-Cam Performs Face Swapping Using the inswapper_128 Model

Deep-Live-Cam performs face swapping by loading the inswapper_128 ONNX model once, then processing each video frame through a pipeline that validates inputs, runs inference with paste_back=True, and applies defensive post-processing to ensure valid output.

Deep-Live-Cam leverages the inswapper_128 model from InsightFace to perform real-time face swapping in video streams. This open-source application implements a robust pipeline that handles model loading, hardware acceleration, and frame processing to replace target faces with source faces seamlessly.

Model Loading and Provider Configuration

The face swap functionality begins with lazy-loading the ONNX model via the get_face_swapper function in modules/processors/frame/face_swapper.py. This function implements a singleton pattern to avoid reloading the model for every frame.

The loader first resolves the appropriate model file—inswapper_128.onnx for general use or inswapper_128_fp16.onnx for CUDA acceleration—then configures the execution provider list. The provider selection logic prioritizes CoreML on Apple Silicon and CUDA when available, falling back to CPU execution if necessary. The function then calls insightface.model_zoo.get_model with the resolved path and provider configuration, caching the result in the global FACE_SWAPPER variable.

The Face Swap Pipeline

Once loaded, the model processes frames through the swap_face function, which implements a three-stage pipeline: pre-processing, inference, and post-processing.

Input Validation and Pre-processing

Before invoking the model, swap_face validates the input frame to ensure it is a uint8 numpy array and converts it to C-contiguous format for optimal ONNX Runtime performance. The function also clamps the opacity setting to valid ranges. If no source face or target face is detected, or if face embeddings are missing, the function returns the original frame unchanged to prevent processing errors.

ONNX Inference with paste_back

The core face swap occurs when the cached face_swapper model is invoked:

swapped_frame_raw = face_swapper.get(
    temp_frame, target_face, source_face, paste_back=True
)

Here, temp_frame represents the current video frame, target_face defines the facial region to replace, and source_face provides the reference embedding from the source image. The paste_back=True parameter instructs the model to render the swapped face directly onto the input frame rather than returning a cropped face patch.

Defensive Post-processing

Following inference, the pipeline implements defensive checks to handle model anomalies. If the output is None or a non-numpy object, the function returns the original frame. When output dimensions mismatch the input, the code attempts GPU-accelerated resizing via gpu_resize. The output values are clipped to the [0, 255] range and cast to uint8 to ensure valid image data.

Optional refinement steps include mouth masking to preserve the original mouth region, Poisson blending for seamless integration, and opacity blending to control the swap intensity.

Implementation Workflow

To implement the face swap in your own code using Deep-Live-Cam's architecture:

# 1. Ensure the model file is present (downloaded automatically)

from modules.utilities import conditional_download
conditional_download(
    models_dir,
    ["https://huggingface.co/hacksider/deep-live-cam/resolve/main/inswapper_128_fp16.onnx"]
)

# 2. Load (or retrieve cached) the swapper

from modules.processors.frame.face_swapper import get_face_swapper
face_swapper = get_face_swapper()          # returns an InsightFace model object

# 3. Perform a single-frame swap

from modules.face_analyser import Face   # data class holding bbox, landmarks, embedding…

source = get_one_face(frame_src)          # face to copy from

target = get_one_face(frame_dst)          # face to replace in destination frame

swapped = swap_face(source, target, frame_dst)   # core swap routine

Summary

• Lazy Loading: The inswapper_128 ONNX model is loaded once via get_face_swapper and cached globally to avoid redundant initialization overhead.
• Hardware Awareness: The pipeline automatically selects CoreML for Apple Silicon, CUDA for NVIDIA GPUs (using FP16 weights), or CPU execution based on availability.
• Defensive Processing: The swap_face function validates inputs, handles None returns, manages shape mismatches with gpu_resize, and clips values to prevent corruption.
• Seamless Integration: The paste_back=True parameter ensures the swapped face is rendered directly onto the original frame, with optional Poisson blending and opacity controls for refinement.

Frequently Asked Questions

What is the inswapper_128 model in Deep-Live-Cam?

The inswapper_128 is an ONNX format neural network model provided by InsightFace that performs 128x128 resolution face swapping. In Deep-Live-Cam, it functions as the core inference engine that takes a source face embedding and a target face region, then generates a photorealistic face swap rendered directly onto the video frame.

How does Deep-Live-Cam handle different hardware accelerators?

Deep-Live-Cam dynamically configures execution providers based on the host hardware. When loading the model via get_face_swapper, it prioritizes CoreML on Apple Silicon devices for optimal performance, selects CUDA with FP16 precision for NVIDIA GPUs, and falls back to CPU execution if no accelerator is available. This provider list is passed directly to the ONNX Runtime during model initialization.

What happens if the face swap fails or returns None?

The swap_face function implements defensive programming to handle inference failures. If the face_swapper.get() call returns None or a non-numpy object, the function immediately returns the original unmodified frame. Similarly, if the output dimensions do not match the input frame, the code attempts a GPU-accelerated resize via gpu_resize before proceeding with post-processing, ensuring the pipeline never crashes due to model anomalies.

Where is the face swap logic implemented in the codebase?

The primary implementation resides in modules/processors/frame/face_swapper.py. This file contains the get_face_swapper function for model loading and the swap_face function that orchestrates the entire pipeline. Supporting utilities for model downloading are located in modules/utilities.py, while face analysis and data structures are defined in modules/face_analyser.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}