Deep-Live-Cam Frame Processor Pipeline Architecture: A Three-Layer Technical Breakdown

Deep-Live-Cam implements a modular, three-layer frame processor pipeline that dynamically loads processor modules, validates interface contracts, and executes parallel per-frame operations across CPU, CUDA, or CoreML backends.

The hacksider/Deep-Live-Cam repository structures its video and image processing around a flexible frame processor pipeline architecture designed for extensibility and performance. This pipeline decouples high-level orchestration from low-level frame manipulation, allowing developers to chain operations like face swapping and enhancement through a standardized plugin interface.

The Three-Layer Pipeline Architecture

The frame processor pipeline is organized into distinct layers that separate concerns between CLI/UI handling, module lifecycle management, and concrete image processing algorithms.

Core Orchestration Layer

The entry point resides in modules/core.py, which parses the --frame-processor CLI argument and stores the requested pipeline chain in modules.globals.frame_processors【^1^】. The get_frame_processors_modules() function dynamically imports the specified processors and validates their readiness through two lifecycle hooks:

• pre_check() – Verifies that required model files and dependencies exist before processing begins.
• pre_start() – Loads ONNX models and initializes execution providers (CUDA, CoreML, or CPU)【^2^】.

For video processing, the process_video() function extracts temporary frames and iterates sequentially over all loaded processors, passing the output of each processor as input to the next【^3^】. This sequential chaining allows complex workflows like face_swapper followed by face_enhancer_gpen256.

Processor Manager Layer

Located at modules/processors/frame/core.py, this layer acts as the runtime engine that manages module discovery and parallel execution. The FRAME_PROCESSORS_INTERFACE constant defines a strict contract requiring six specific callables: pre_check, pre_start, process_frame, process_image, process_video, and process_frames【^4^】.

The load_frame_processor_module() function dynamically imports modules.processors.frame.<name> and aborts execution if any interface method is missing【^5^】. Once loaded, get_frame_processors_modules() caches these modules and synchronizes UI-driven state through modules.globals.fp_ui, enabling runtime processor toggling from the graphical interface【^6^】.

Parallel execution is handled by multi_process_frame(), which splits temporary frame paths into batches and distributes them across a ThreadPoolExecutor. The batch size auto-tunes based on modules.globals.execution_threads and available system memory【^7^】.

Individual Processor Modules

Concrete implementations reside in modules/processors/frame/ and follow a standardized skeleton. Each processor must expose the interface contract methods while implementing domain-specific logic:

• face_swapper.py – Loads InsightFace ONNX models and performs face swapping frame-by-frame, with optional CoreML acceleration on Apple silicon【^8^】.
• face_enhancer_gpen256.py and face_enhancer_gpen512.py – Apply GPEN-based super-resolution networks at 256× or 512× resolution to detected face regions【^9^】.

The process_frame() method receives a source face object and a temporary frame, returning the modified frame. For video workflows, process_video() orchestrates calls to multi_process_frame() from the core layer, while process_image() handles single-image I/O operations.

The Processor Interface Contract

Every frame processor must implement the six-function contract defined in FRAME_PROCESSORS_INTERFACE within modules/processors/frame/core.py:

def pre_check() -> bool:
    """Verify model files and dependencies are available."""
    
def pre_start() -> bool:
    """Initialize models and execution providers."""
    
def process_frame(source_face: Any, temp_frame: Frame) -> Frame:
    """Transform a single frame (core processing logic)."""
    
def process_image(source_path: str, target_path: str, output_path: str) -> None:
    """Handle image-to-image processing workflow."""
    
def process_video(source_path: str, temp_frame_paths: List[str]) -> None:
    """Orchestrate video frame processing using multi_process_frame."""

The load_frame_processor_module() function uses importlib to dynamically load these modules and validates that all six functions exist via inspect checks【^5^】. This contract ensures that the core orchestration layer can treat all processors—whether built-in or third-party—as interchangeable components.

Parallel Execution and Batching

The pipeline achieves performance through intra-processor parallelism controlled by the processor manager layer. When process_video() invokes a processor, the implementation typically delegates to multi_process_frame():

# Simplified flow from modules/processors/frame/core.py

def multi_process_frame(source_path, temp_frame_paths, process_frames):
    # Split frames into batches based on execution_threads

    batches = create_batches(temp_frame_paths, modules.globals.execution_threads)
    
    with ThreadPoolExecutor() as executor:
        executor.map(process_frames, batches)

Each processor receives the entire list of temporary frame paths but processes them concurrently within its own process_frames callback. This design allows CPU-intensive enhancers and GPU-accelerated swappers to utilize separate thread pools without blocking the main execution flow【^7^】.

Extending the Pipeline with Custom Processors

Adding new functionality requires creating a Python file in modules/processors/frame/ and implementing the six-function interface. For example, a custom color grading processor would look like:

# modules/processors/frame/color_grader.py

import cv2
from typing import List, Any

NAME = "COLOR_GRADER"

def pre_check() -> bool:
    return True  # No external assets required

def pre_start() -> bool:
    return True

def process_frame(source_face: Any, temp_frame: Any) -> Any:
    # Increase brightness by 30 units

    return cv2.convertScaleAbs(temp_frame, alpha=1.0, beta=30)

def process_image(source_path: str, target_path: str, output_path: str) -> None:
    frame = cv2.imread(target_path)
    result = process_frame(None, frame)
    cv2.imwrite(output_path, result)

def process_video(source_path: str, temp_frame_paths: List[str]) -> None:
    from modules.processors.frame.core import multi_process_frame
    multi_process_frame(
        source_path, 
        temp_frame_paths, 
        lambda src, paths, progress: process_frame(src, cv2.imread(paths[0]))
    )

def process_frames(src: Any, paths: List[str], progress: Any = None) -> None:
    return process_frame(src, cv2.imread(paths[0]))

To activate the processor, add its module name to the --frame-processor CLI argument or update the choices list in modules/core.py where the argument parser defines choices=['face_swapper', 'face_enhancer_gpen256', 'color_grader'].

Code Examples

Running Multiple Processors via CLI

Execute a chained pipeline that swaps faces then enhances them using the 256-pixel GPEN model:

python -m modules.run \
    -s source.jpg \
    -t target.mp4 \
    -o output.mp4 \
    --frame-processor face_swapper face_enhancer_gpen256 \
    --execution-provider cuda \
    --execution-threads 4

The execution order follows the CLI argument sequence—face_swapper processes all frames first, then face_enhancer_gpen256 receives the swapped frames as input.

Programmatic Pipeline Configuration

Configure and run the pipeline from within a Python script without CLI parsing:

from modules import core, globals, processors

# Configure runtime globals (equivalent to CLI arguments)

globals.source_path = "source.jpg"
globals.target_path = "target.png"
globals.output_path = "output.png"
globals.frame_processors = ["face_swapper", "face_enhancer_gpen256"]
globals.execution_providers = ["cpu"]
globals.execution_threads = 4

# Initialize all requested processors

processor_modules = core.get_frame_processors_modules(globals.frame_processors)
for processor in processor_modules:
    assert processor.pre_check(), "Pre-check failed"
    assert processor.pre_start(), "Pre-start failed"

# Execute image processing

processor_modules[0].process_image(
    globals.source_path, 
    globals.target_path, 
    globals.output_path
)

Summary

• Three-layer architecture separates CLI orchestration (modules/core.py), module management (modules/processors/frame/core.py), and concrete processing algorithms.
• Strict interface contract requires six specific functions (pre_check, pre_start, process_frame, process_image, process_video, process_frames) for all processor modules.
• Dynamic loading via load_frame_processor_module() enables runtime processor selection through the --frame-processor argument.
• Parallel execution is handled by multi_process_frame() using ThreadPoolExecutor with auto-tuned batch sizes based on execution_threads.
• Sequential chaining allows multiple processors to feed output from one stage into the next, enabling complex video enhancement workflows.

Frequently Asked Questions

How do I add a custom frame processor to Deep-Live-Cam?

Create a new Python file in modules/processors/frame/ implementing the six-function interface contract defined in FRAME_PROCESSORS_INTERFACE. Include pre_check(), pre_start(), and processing functions, then reference the module name in the --frame-processor CLI argument. The processor manager layer will automatically import and validate your module at runtime.

What is the execution order when multiple frame processors are specified?

Processors execute sequentially in the exact order provided to the --frame-processor argument. For example, --frame-processor face_swapper face_enhancer_gpen256 runs the swapper on all frames first, then passes those modified frames to the enhancer. This chaining happens inside process_video() in modules/core.py.

How does Deep-Live-Cam handle parallel processing of video frames?

Each processor utilizes multi_process_frame() from modules/processors/frame/core.py, which splits temporary frame paths into batches and distributes them across a ThreadPoolExecutor. The batch size scales automatically based on the execution_threads global setting, allowing concurrent frame processing within each pipeline stage without blocking the main thread.

What interface methods must a frame processor implement?

Every processor must implement six methods: pre_check() for dependency validation, pre_start() for model initialization, process_frame() for single-frame transformation, process_image() for image file I/O, process_video() for video orchestration, and process_frames() as the batch processing callback. The load_frame_processor_module() function validates these exist before execution begins.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}