# How to Resolve Kernel Registration Conflicts with Multiple Execution Providers in ONNX Runtime

> Resolve ONNX Runtime kernel registration conflicts with multiple execution providers. Learn how to ensure unique types, avoid version overlaps, and register providers correctly.

- Repository: [Microsoft/onnxruntime](https://github.com/microsoft/onnxruntime)
- Tags: how-to-guide
- Published: 2026-04-24

---

**To resolve kernel registration conflicts when using multiple execution providers in ONNX Runtime, ensure each execution provider implements a unique type string via `IExecutionProvider::Type()`, avoid overlapping operator version ranges when calling `KernelRegistry::Register`, and register exactly one instance per provider type in the `KernelRegistryManager`.**

When building high-performance inference pipelines with the `microsoft/onnxruntime` repository, developers often combine multiple execution providers (EPs) such as CUDA, TensorRT, and custom accelerators. However, the framework strictly enforces uniqueness constraints in its kernel registry system, throwing errors when duplicate provider types or conflicting operator versions are detected. Understanding the internal mechanics of `KernelRegistryManager` and `KernelRegistry` is essential for debugging these issues and maintaining stable multi-EP sessions.

## Understanding the Kernel Registration Architecture

ONNX Runtime discovers operator implementations (kernels) through a **kernel registry** built for each execution provider. The `KernelRegistryManager` maintains a mapping between provider types and their respective registries.

### How the Registry Manager Organizes Providers

When you attach execution providers to a session, the runtime invokes `KernelRegistryManager::RegisterKernels` (implemented in `onnxruntime/core/framework/kernel_registry_manager.cc` at lines 33-48). This method iterates through the execution provider list and inserts each provider's kernel registry into an internal map keyed by the string returned from `IExecutionProvider::Type()`.

The manager validates that no duplicate provider types exist. If the same type string appears twice, the runtime immediately throws:

```

found duplicated provider <EP_TYPE> in KernelRegistryManager

```

### The Two Conflict Categories

Kernel registration conflicts fall into two distinct categories checked at different layers:

1. **Provider-level duplicates**: Occurs in `KernelRegistryManager::RegisterKernels` when two EP instances report identical type strings.
2. **Operator-level conflicts**: Occurs in `KernelRegistry::Register` (lines 12-26 of `onnxruntime/core/framework/kernel_registry.cc`) when two kernels for the same operator type have overlapping version ranges within the same registry.

The second check calls `KernelDef::IsConflict` to detect overlapping version ranges, producing the error:

```

Failed to add kernel for <key>: Conflicting with a registered kernel with op versions …

```

## Diagnosing Kernel Registration Conflicts

When debugging registration failures, examine the stack trace to identify which validation failed. In `kernel_registry_manager.cc`, the duplicate provider check uses a simple map insertion:

```cpp
for (auto& ep : execution_providers) {
  auto it = provider_type_to_registry_.find(ep->Type());
  if (it != provider_type_to_registry_.end()) {
    LOGS_DEFAULT(ERROR) << "Duplicate EP type: " << ep->Type();
    return Status(common::ONNXRUNTIME, common::FAIL, "found duplicated provider...");
  }
}

```

For operator conflicts, the validation occurs inside `KernelRegistry::Register` at lines 15-23, where the registry checks if any existing kernel definition covers the same operator type and version range.

## Resolving Duplicate Provider Type Conflicts

### The Single Instance Pattern

The most common cause of provider-level conflicts is creating multiple instances of the same execution provider class. The runtime architecture expects **exactly one** instance of each EP type per session.

If you need separate configurations for the same provider type (e.g., two different GPU devices), you must create distinct provider types by deriving a custom class and overriding the `Type()` method:

```cpp
class CudaDevice0EP : public CudaExecutionProvider {
 public:
  CudaDevice0EP() : CudaExecutionProvider{/*options*/} {}
  const std::string& Type() const override {
    static const std::string type = "CUDA0";
    return type;
  }
};

class CudaDevice1EP : public CudaExecutionProvider {
 public:
  CudaDevice1EP() : CudaExecutionProvider{/*options*/} {}
  const std::string& Type() const override {
    static const std::string type = "CUDA1";
    return type;
  }
};

```

Both objects can now be added to the same session without triggering the duplicate provider error, as `KernelRegistryManager` stores them under distinct keys (`"CUDA0"` and `"CUDA1"`).

### Custom Execution Provider Naming

When implementing custom execution providers, always choose unique type strings that do not collide with built-in providers (`"CPU"`, `"CUDA"`, `"TensorRT"`, etc.):

```cpp
class MyCustomExecutionProvider : public IExecutionProvider {
 public:
  MyCustomExecutionProvider() : IExecutionProvider{"MyCustom"} {}  // Unique type
  std::shared_ptr<KernelRegistry> GetKernelRegistry() const override {
    static std::shared_ptr<KernelRegistry> registry = []() {
      auto r = std::make_shared<KernelRegistry>();
      // Register kernels with explicit provider attribution
      ORT_RETURN_IF_ERROR(r->Register(
          KernelDefBuilder()
              .SetName("MyOp")
              .SinceVersion(1)
              .Provider("MyCustom"),  // Matches the EP type
          [](const OpKernelInfo& info, std::unique_ptr<OpKernel>& out) {
            out = std::make_unique<MyOpKernel>(info);
            return Status::OK();
          }));
      return r;
    }();
    return registry;
  }
};

```

The unique provider name `"MyCustom"` ensures that `KernelRegistryManager::RegisterKernels` never encounters a duplicate key.

## Resolving Operator Version Conflicts

### Defining Non-Overlapping Version Ranges

When registering custom kernels that might overlap with built-in implementations, carefully specify version ranges using `KernelDefBuilder::SinceVersion(start, end)`. Overlapping ranges for the same operator type within a single registry trigger the conflict error:

```cpp
KernelDefBuilder builder;
builder.SetName("Add")
       .SinceVersion(1, 12)          // Supports versions 1-12 only
       .Provider(kCudaExecutionProvider);

auto status = registry->Register(builder, CreateMyAddKernel);
if (!status.IsOK()) {
  LOGS_DEFAULT(ERROR) << "Kernel registration failure: " << status.ErrorMessage();
}

```

If another kernel already covers versions 1-12 in the same registry, `KernelRegistry::Register` returns a failure status.

### Using Custom Kernel Registries

To augment existing kernels without modifying built-in registries, create a separate registry and register it with `RegisterKernelRegistry`. The manager searches `custom_kernel_registries_` (see `SearchKernelRegistry` around line 84 of `kernel_registry_manager.cc`) before checking built-in registries:

```cpp
auto custom_registry = std::make_shared<KernelRegistry>();
// Register your custom kernels here
kernel_registry_manager.RegisterKernelRegistry(custom_registry);

```

This approach allows you to provide specialized implementations that take precedence over default kernels while avoiding version conflicts in the built-in CUDA or CPU registries.

## Preventing Plugin Double-Registration

When loading execution providers as plugins, guard registration code with `std::call_once` to ensure kernels are registered exactly once:

```cpp
std::once_flag registration_flag;

void RegisterCustomKernels(KernelRegistry& registry) {
  std::call_once(registration_flag, [&]() {
    // Registration logic here
  });
}

```

This prevents the "Failed to add kernel" error when the same plugin is inadvertently loaded multiple times.

## Summary

- **Ensure unique provider types** by overriding `IExecutionProvider::Type()` when creating multiple instances of similar EPs or custom providers.
- **Avoid version range overlaps** when calling `KernelRegistry::Register` by using explicit `SinceVersion(start, end)` boundaries.
- **Register custom kernels separately** using `KernelRegistryManager::RegisterKernelRegistry` to inject implementations without touching built-in registries.
- **Use single-instance patterns** for EP creation, or derive custom classes with distinct type strings for multiple device configurations.
- **Guard plugin registration** with static initialization or `std::call_once` to prevent duplicate kernel insertion.

## Frequently Asked Questions

### What causes the "found duplicated provider" error in ONNX Runtime?

This error occurs in `KernelRegistryManager::RegisterKernels` (lines 33-48 of `kernel_registry_manager.cc`) when two execution provider instances return the same string from `IExecutionProvider::Type()`. The manager maintains a map keyed by provider type strings and rejects duplicate keys. Create only one instance per EP type, or override `Type()` to return unique identifiers for distinct configurations.

### How do I register kernels for multiple GPU devices without conflicts?

Derive separate execution provider classes from the base CUDA provider and override the `Type()` method to return distinct strings (e.g., `"CUDA0"` and `"CUDA1"`). According to the `microsoft/onnxruntime` source code, the `KernelRegistryManager` stores registries in a map keyed by these type strings, so unique identifiers eliminate duplicate provider conflicts while allowing multiple CUDA EPs in one session.

### Why does my custom kernel conflict with built-in implementations even with different provider types?

If you receive the error "Failed to add kernel for ... Conflicting with a registered kernel," the conflict occurs within a single `KernelRegistry` instance, not across the manager. Check that your kernel's version range (defined via `KernelDefBuilder::SinceVersion`) does not overlap with existing registrations in that specific registry. Alternatively, register your kernels in a separate registry using `RegisterKernelRegistry`, which is searched before built-in registries according to `SearchKernelRegistry` in `kernel_registry_manager.cc`.

### Can I replace a built-in kernel with my custom implementation?

Yes, but you must avoid version conflicts in the same registry. The cleanest approach is to create a custom `KernelRegistry` containing your replacement kernels and register it via `KernelRegistryManager::RegisterKernelRegistry`. Because the manager searches custom registries before built-in ones (around line 84 of `kernel_registry_manager.cc`), your implementation will be selected first without needing to unregister the original.