# ONNX Runtime Multi-Provider Configuration in Deep-Live-Cam: A Complete Guide

> Learn to configure ONNX Runtime multi-provider usage in Deep-Live-Cam. Decode CLI args, store providers, and optimize hardware settings for enhanced performance.

- Repository: [Kenneth Estanislao/Deep-Live-Cam](https://github.com/hacksider/Deep-Live-Cam)
- Tags: deep-dive
- Published: 2026-03-01

---

**Deep-Live-Cam configures ONNX Runtime for multi-provider usage by decoding CLI arguments into provider strings via `core.decode_execution_providers`, storing them in `modules.globals.execution_providers`, and passing them to `InferenceSession` constructors—optionally wrapping specific providers in tuples with options dictionaries for hardware-specific optimizations like CoreML's Neural Engine settings.**

Deep-Live-Cam leverages ONNX Runtime to execute face-swapping and enhancement models across diverse hardware accelerators including CUDA, CoreML, DirectML, and ROCm. Understanding how to configure ONNX Runtime multi-provider usage with provider-specific options allows you to optimize inference performance by enabling fallback chains and hardware-specific optimizations.

## How Provider Selection Works in Deep-Live-Cam

The application implements a two-stage pipeline for configuring ONNX Runtime execution providers. First, it translates user-friendly CLI arguments into ONNX Runtime provider strings. Second, it optionally constructs provider-specific configuration dictionaries before passing them to session constructors.

### Decoding Execution Providers from CLI Arguments

When you launch Deep-Live-Cam with the `--execution-provider` flag, the `core.decode_execution_providers` function in [`modules/core.py`](https://github.com/hacksider/Deep-Live-Cam/blob/main/modules/core.py) maps short names like `cuda` or `coreml` to their full ONNX Runtime identifiers. This function filters `onnxruntime.get_available_providers()` against your requested providers and returns a prioritized list.

```python

# modules/core.py – lines 119-122

def decode_execution_providers(execution_providers: List[str]) -> List[str]:
    return [
        provider
        for provider, encoded_execution_provider in zip(
            onnxruntime.get_available_providers(),
            encode_execution_providers(onnxruntime.get_available_providers())
        )
        if any(execution_provider in encoded_execution_provider
               for execution_provider in execution_providers)
    ]

```

The resulting list is stored in `modules.globals.execution_providers`, making it available to all frame processors throughout the application lifecycle.

### Global Provider Storage

The `execution_providers` variable in [`modules/globals.py`](https://github.com/hacksider/Deep-Live-Cam/blob/main/modules/globals.py) acts as the single source of truth for ONNX Runtime configuration across the application. Every processor that initializes an ONNX session reads from this global list, ensuring consistent provider usage whether you are running the face swapper, face enhancer, or other modules.

## Configuring Provider-Specific Options

While many processors pass the provider list directly to `InferenceSession`, some require provider-specific options to unlock hardware acceleration features. Deep-Live-Cam implements this by constructing a **provider-config list**, where items are either plain provider strings or tuples containing `(provider_name, options_dict)`.

### CoreML Configuration on Apple Silicon

The face swapper processor in [`modules/processors/frame/face_swapper.py`](https://github.com/hacksider/Deep-Live-Cam/blob/main/modules/processors/frame/face_swapper.py) demonstrates advanced multi-provider configuration. When `CoreMLExecutionProvider` is selected and the system runs on Apple Silicon, it constructs a detailed options dictionary to enable the Neural Engine, GPU, and CPU simultaneously:

```python

# modules/processors/frame/face_swapper.py – lines 94-118

providers_config = []
for p in modules.globals.execution_providers:
    if p == "CoreMLExecutionProvider" and IS_APPLE_SILICON:
        providers_config.append((
            "CoreMLExecutionProvider",
            {
                "ModelFormat": "MLProgram",
                "MLComputeUnits": "ALL",                  # NE + GPU + CPU

                "SpecializationStrategy": "FastPrediction",
                "AllowLowPrecisionAccumulationOnGPU": 1,
                "EnableOnSubgraphs": 1,
                "RequireStaticShapes": 0,
                "MaximumCacheSize": 1024 * 1024 * 512,   # 512 MiB

            },
        ))
    else:
        providers_config.append(p)

FACE_SWAPPER = insightface.model_zoo.get_model(
    model_path,
    providers=providers_config,
)

```

This pattern allows ONNX Runtime to leverage Apple-specific optimizations while maintaining compatibility with other providers in the fallback chain.

### Generic Session Creation with SessionOptions

Other processors like the GFPGAN face enhancer use a simpler approach with `SessionOptions` for graph optimization while passing the standard provider list:

```python

# modules/processors/frame/face_enhancer.py – lines 84-95

session_options = onnxruntime.SessionOptions()
session_options.graph_optimization_level = onnxruntime.GraphOptimizationLevel.ORT_ENABLE_ALL

FACE_ENHANCER = onnxruntime.InferenceSession(
    model_path,
    sess_options=session_options,
    providers=modules.globals.execution_providers,
)

```

## Practical Configuration Examples

### CLI Multi-Provider Setup

To configure multiple providers with automatic fallback ordering, pass space-separated values to `--execution-provider`. ONNX Runtime attempts providers in the order specified, falling back to the next if the current fails:

```bash
python run.py \
    -s source.jpg \
    -t target.mp4 \
    --execution-provider cuda cpu \
    --frame-processor face_swapper face_enhancer

```

This configuration attempts CUDA first, then falls back to CPU if the CUDA provider encounters an error or is unavailable.

### Programmatic Configuration

For custom integrations or Jupyter notebooks, you can manually set the global providers before initializing processors:

```python
import modules.globals as gl
import modules.core as core
import onnxruntime as ort

# Decode and set providers

gl.execution_providers = core.decode_execution_providers(['cuda', 'cpu'])

# Create session with global configuration

session = ort.InferenceSession(
    "models/gfpgan-1024.onnx",
    providers=gl.execution_providers,
    sess_options=ort.SessionOptions()
)
print("Active providers:", session.get_providers())

```

### Adding Custom Provider Options

To inject provider-specific options for unsupported providers like DirectML, modify the global list before processor initialization:

```python
import modules.globals as gl

# Set base providers

gl.execution_providers = ["DmlExecutionProvider", "CPUExecutionProvider"]

# Wrap DirectML with custom memory options

gl.execution_providers = [
    ("DmlExecutionProvider", {
        "arena_extend_strategy": "kSameAsRequested",
        "device_id": 0,
    }),
    "CPUExecutionProvider"
]

```

## Summary

- **Provider decoding** happens in [`modules/core.py`](https://github.com/hacksider/Deep-Live-Cam/blob/main/modules/core.py) via `decode_execution_providers`, mapping CLI short names to ONNX Runtime identifiers.
- **Global state** is maintained in `modules/globals.execution_providers`, accessible to all processors.
- **Provider-specific options** are implemented as tuples `(provider_name, options_dict)` in the provider list, as demonstrated in [`modules/processors/frame/face_swapper.py`](https://github.com/hacksider/Deep-Live-Cam/blob/main/modules/processors/frame/face_swapper.py) for CoreML.
- **Session creation** varies by processor: some use plain provider lists while others combine `SessionOptions` with provider configurations.
- **Fallback behavior** follows the list order; ONNX Runtime attempts each provider sequentially until one succeeds.

## Frequently Asked Questions

### How do I enable CoreML with Neural Engine support on Apple Silicon?

Pass `coreml` to the `--execution-provider` flag. The face swapper processor in [`modules/processors/frame/face_swapper.py`](https://github.com/hacksider/Deep-Live-Cam/blob/main/modules/processors/frame/face_swapper.py) automatically detects Apple Silicon and constructs a provider-specific options dictionary setting `MLComputeUnits` to `"ALL"`, enabling the Neural Engine, GPU, and CPU simultaneously.

### Can I mix different provider types in the same command?

Yes. Deep-Live-Cam supports multi-provider chains like `--execution-provider cuda coreml cpu`. The `decode_execution_providers` function filters and orders providers based on availability, and ONNX Runtime falls back through the list if a provider fails during inference.

### Where is the execution provider configuration stored globally?

The `execution_providers` list in [`modules/globals.py`](https://github.com/hacksider/Deep-Live-Cam/blob/main/modules/globals.py) stores the decoded provider strings. This global variable is populated during CLI argument parsing in [`modules/core.py`](https://github.com/hacksider/Deep-Live-Cam/blob/main/modules/core.py) and read by processors in `modules/processors/frame/` when constructing `InferenceSession` instances.

### How do I add custom options for CUDA or DirectML providers?

Modify `modules.globals.execution_providers` programmatically before initializing processors. Replace the provider string with a tuple containing the provider name and options dictionary: `("CUDAExecutionProvider", {"cudnn_conv_algo_search": "HEURISTIC"})`. The face swapper's CoreML implementation provides the template for this pattern.