How Deep-Live-Cam Selects and Configures Execution Providers (CUDA, DirectML, OpenVINO)

Deep-Live-Cam uses ONNX Runtime as the inference engine and selects execution providers through a validation pipeline that maps CLI arguments like cuda or directml to their full ONNX names, storing them in a global list that every model session consumes.

Deep-Live-Cam leverages ONNX Runtime to run AI face-swapping models efficiently across diverse hardware. The application abstracts GPU acceleration through a unified execution provider interface, allowing users to specify backends via command-line arguments without modifying the underlying model code.

Execution Provider Selection Pipeline

Deep-Live-Cam follows a six-step pipeline to resolve, validate, and apply execution providers across the application.

CLI Argument Parsing

In modules/core.py, the parse_args() function defines the --execution-provider argument with a default value of cpu. The function calls suggest_execution_providers() to dynamically populate the list of valid choices based on the host system's available ONNX Runtime providers (lines 55-57).

Validation and Name Encoding

The decode_execution_providers() function in modules/core.py (lines 19-22) performs case-insensitive matching of user inputs against the providers returned by onnxruntime.get_available_providers(). It converts short names like cuda into their canonical forms—CUDAExecutionProvider—using the encode_execution_providers() helper.

Global State Management

Once validated, the resolved provider list is stored in modules.globals.execution_providers (defined at line 47 in modules/globals.py). This mutable list acts as the single source of truth for all ONNX sessions throughout the application.

Session Initialization

When creating inference sessions, processors pass this global list directly to ONNX Runtime. In modules/processors/frame/_onnx_enhancer.py, the create_onnx_session() method instantiates onnxruntime.InferenceSession with the global providers (lines 24-28). The same pattern appears in modules/processors/frame/face_enhancer.py within the get_face_enhancer() function.

Thread Count Heuristics

Deep-Live-Cam automatically adjusts CPU thread allocation based on the selected provider. The suggest_execution_threads() function in modules/core.py (lines 41-48) returns 1 when GPU-based providers like CUDA, DirectML, or ROCm are active, since the GPU handles parallelization. For pure CPU execution, it may suggest up to 16 threads.

Environment Variable Tweaks

To prevent performance penalties during CUDA operations, the application forces OMP_NUM_THREADS=1 at the top of modules/core.py (lines 4-6) before importing torch if any execution provider is specified. This prevents OpenMP thread contention with GPU inference.

Supported Execution Providers and Installation

Deep-Live-Cam supports multiple hardware accelerators through specific ONNX Runtime wheels. The availability of each provider depends on the installed package and system drivers.

• CUDA (NVIDIA): Requires onnxruntime-gpu==1.21.0 and NVIDIA drivers compatible with CUDA 12.8 and cuDNN 8.9.7. Maps to CUDAExecutionProvider.
• DirectML (Windows): Requires onnxruntime-directml==1.21.0. Detected on Windows systems with DirectML support as DmlExecutionProvider.
• OpenVINO (Intel): Requires onnxruntime-openvino==1.21.0. Provides OpenVINOExecutionProvider for Intel CPUs and integrated GPUs.
• CoreML (Apple Silicon): Requires onnxruntime-silicon==1.13.1. Enables CoreMLExecutionProvider on macOS ARM64 devices.
• CPU (Fallback): Always available via CPUExecutionProvider without additional installation.

The provider list maintains the exact order specified by the user, enabling fallback chains (e.g., cuda cpu). ONNX Runtime attempts to initialize the first provider; if it fails, it silently proceeds to the next.

Practical Usage Examples

Command Line Execution

Specify providers using the --execution-provider flag:

# NVIDIA CUDA acceleration

python run.py --execution-provider cuda

# Windows DirectML

python run.py --execution-provider directml

# Intel OpenVINO

python run.py --execution-provider openvino

# GPU with CPU fallback chain

python run.py --execution-provider cuda cpu

Programmatic Provider Selection

Override the global configuration in Python to force a specific provider without CLI arguments:

import onnxruntime
import modules.globals as G

# Force OpenVINO execution

G.execution_providers = ['OpenVINOExecutionProvider']

# Create session with selected provider

session = onnxruntime.InferenceSession(
    'models/face_enhancer.onnx',
    providers=G.execution_providers
)

Runtime Inspection

Verify active providers during execution:

from modules import globals as G
print("Active providers:", G.execution_providers)

# Output: ['CUDAExecutionProvider']

Checking Thread Configuration

Inspect the suggested thread count for your current hardware:

from modules.core import suggest_execution_threads
print("Execution threads:", suggest_execution_threads())

# Returns 1 for GPU providers, higher for CPU-only

Summary

• Deep-Live-Cam delegates all hardware acceleration to ONNX Runtime through a provider abstraction layer implemented in modules/core.py.
• Provider selection occurs via CLI parsing, case-insensitive validation against onnxruntime.get_available_providers(), and global storage in modules/globals.execution_providers (line 47).
• Supported providers include CUDA, DirectML, OpenVINO, CoreML, and CPU, each requiring specific onnxruntime-* wheel installations as documented in the README.
• Inference sessions in modules/processors/frame/_onnx_enhancer.py and face_enhancer.py automatically consume the global provider list during InferenceSession creation.
• The application optimizes performance by setting single-thread CPU mode when GPU providers are active and forcing OMP_NUM_THREADS=1 before torch initialization to prevent thread contention.

Frequently Asked Questions

How do I know which execution providers are available on my system?

Deep-Live-Cam queries ONNX Runtime's get_available_providers() method during startup to populate the allowed CLI choices. You can verify availability manually by running python -c "import onnxruntime; print(onnxruntime.get_available_providers())" after installing the appropriate wheel (e.g., onnxruntime-gpu for CUDA).

Can I use multiple execution providers at once?

Yes. Pass a space-separated list to --execution-provider, such as cuda cpu. According to the source code in modules/core.py, ONNX Runtime attempts to initialize the first provider; if it fails (e.g., due to missing GPU drivers), it silently falls back to the next provider in the ordered list.

Why does Deep-Live-Cam limit CPU threads when using CUDA or DirectML?

The suggest_execution_threads() function in modules/core.py (lines 41-48) returns 1 for GPU providers because the GPU already handles massive parallelization. Limiting CPU threads prevents thread contention and reduces overhead, improving overall inference latency for the face-swapping pipeline.

Where is the execution provider configuration stored during runtime?

The validated list of full provider names (e.g., ['CUDAExecutionProvider']) is stored in the mutable list modules.globals.execution_providers, defined at line 47 of modules/globals.py. All ONNX session creators in the processor modules reference this global variable to ensure consistent hardware acceleration across the application.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}