How to Resolve Kernel Registration Conflicts with Multiple Execution Providers in ONNX Runtime

To resolve kernel registration conflicts when using multiple execution providers in ONNX Runtime, ensure each execution provider implements a unique type string via IExecutionProvider::Type(), avoid overlapping operator version ranges when calling KernelRegistry::Register, and register exactly one instance per provider type in the KernelRegistryManager.

When building high-performance inference pipelines with the microsoft/onnxruntime repository, developers often combine multiple execution providers (EPs) such as CUDA, TensorRT, and custom accelerators. However, the framework strictly enforces uniqueness constraints in its kernel registry system, throwing errors when duplicate provider types or conflicting operator versions are detected. Understanding the internal mechanics of KernelRegistryManager and KernelRegistry is essential for debugging these issues and maintaining stable multi-EP sessions.

Understanding the Kernel Registration Architecture

ONNX Runtime discovers operator implementations (kernels) through a kernel registry built for each execution provider. The KernelRegistryManager maintains a mapping between provider types and their respective registries.

How the Registry Manager Organizes Providers

When you attach execution providers to a session, the runtime invokes KernelRegistryManager::RegisterKernels (implemented in onnxruntime/core/framework/kernel_registry_manager.cc at lines 33-48). This method iterates through the execution provider list and inserts each provider's kernel registry into an internal map keyed by the string returned from IExecutionProvider::Type().

The manager validates that no duplicate provider types exist. If the same type string appears twice, the runtime immediately throws:

found duplicated provider <EP_TYPE> in KernelRegistryManager

The Two Conflict Categories

Kernel registration conflicts fall into two distinct categories checked at different layers:

1. Provider-level duplicates: Occurs in KernelRegistryManager::RegisterKernels when two EP instances report identical type strings.
2. Operator-level conflicts: Occurs in KernelRegistry::Register (lines 12-26 of onnxruntime/core/framework/kernel_registry.cc) when two kernels for the same operator type have overlapping version ranges within the same registry.

The second check calls KernelDef::IsConflict to detect overlapping version ranges, producing the error:

Failed to add kernel for <key>: Conflicting with a registered kernel with op versions …

Diagnosing Kernel Registration Conflicts

When debugging registration failures, examine the stack trace to identify which validation failed. In kernel_registry_manager.cc, the duplicate provider check uses a simple map insertion:

for (auto& ep : execution_providers) {
  auto it = provider_type_to_registry_.find(ep->Type());
  if (it != provider_type_to_registry_.end()) {
    LOGS_DEFAULT(ERROR) << "Duplicate EP type: " << ep->Type();
    return Status(common::ONNXRUNTIME, common::FAIL, "found duplicated provider...");
  }
}

For operator conflicts, the validation occurs inside KernelRegistry::Register at lines 15-23, where the registry checks if any existing kernel definition covers the same operator type and version range.

Resolving Duplicate Provider Type Conflicts

The Single Instance Pattern

The most common cause of provider-level conflicts is creating multiple instances of the same execution provider class. The runtime architecture expects exactly one instance of each EP type per session.

If you need separate configurations for the same provider type (e.g., two different GPU devices), you must create distinct provider types by deriving a custom class and overriding the Type() method:

class CudaDevice0EP : public CudaExecutionProvider {
 public:
  CudaDevice0EP() : CudaExecutionProvider{/*options*/} {}
  const std::string& Type() const override {
    static const std::string type = "CUDA0";
    return type;
  }
};

class CudaDevice1EP : public CudaExecutionProvider {
 public:
  CudaDevice1EP() : CudaExecutionProvider{/*options*/} {}
  const std::string& Type() const override {
    static const std::string type = "CUDA1";
    return type;
  }
};

Both objects can now be added to the same session without triggering the duplicate provider error, as KernelRegistryManager stores them under distinct keys ("CUDA0" and "CUDA1").

Custom Execution Provider Naming

When implementing custom execution providers, always choose unique type strings that do not collide with built-in providers ("CPU", "CUDA", "TensorRT", etc.):

class MyCustomExecutionProvider : public IExecutionProvider {
 public:
  MyCustomExecutionProvider() : IExecutionProvider{"MyCustom"} {}  // Unique type
  std::shared_ptr<KernelRegistry> GetKernelRegistry() const override {
    static std::shared_ptr<KernelRegistry> registry = []() {
      auto r = std::make_shared<KernelRegistry>();
      // Register kernels with explicit provider attribution
      ORT_RETURN_IF_ERROR(r->Register(
          KernelDefBuilder()
              .SetName("MyOp")
              .SinceVersion(1)
              .Provider("MyCustom"),  // Matches the EP type
          [](const OpKernelInfo& info, std::unique_ptr<OpKernel>& out) {
            out = std::make_unique<MyOpKernel>(info);
            return Status::OK();
          }));
      return r;
    }();
    return registry;
  }
};

The unique provider name "MyCustom" ensures that KernelRegistryManager::RegisterKernels never encounters a duplicate key.

Resolving Operator Version Conflicts

Defining Non-Overlapping Version Ranges

When registering custom kernels that might overlap with built-in implementations, carefully specify version ranges using KernelDefBuilder::SinceVersion(start, end). Overlapping ranges for the same operator type within a single registry trigger the conflict error:

KernelDefBuilder builder;
builder.SetName("Add")
       .SinceVersion(1, 12)          // Supports versions 1-12 only
       .Provider(kCudaExecutionProvider);

auto status = registry->Register(builder, CreateMyAddKernel);
if (!status.IsOK()) {
  LOGS_DEFAULT(ERROR) << "Kernel registration failure: " << status.ErrorMessage();
}

If another kernel already covers versions 1-12 in the same registry, KernelRegistry::Register returns a failure status.

Using Custom Kernel Registries

To augment existing kernels without modifying built-in registries, create a separate registry and register it with RegisterKernelRegistry. The manager searches custom_kernel_registries_ (see SearchKernelRegistry around line 84 of kernel_registry_manager.cc) before checking built-in registries:

auto custom_registry = std::make_shared<KernelRegistry>();
// Register your custom kernels here
kernel_registry_manager.RegisterKernelRegistry(custom_registry);

This approach allows you to provide specialized implementations that take precedence over default kernels while avoiding version conflicts in the built-in CUDA or CPU registries.

Preventing Plugin Double-Registration

When loading execution providers as plugins, guard registration code with std::call_once to ensure kernels are registered exactly once:

std::once_flag registration_flag;

void RegisterCustomKernels(KernelRegistry& registry) {
  std::call_once(registration_flag, [&]() {
    // Registration logic here
  });
}

This prevents the "Failed to add kernel" error when the same plugin is inadvertently loaded multiple times.

Summary

• Ensure unique provider types by overriding IExecutionProvider::Type() when creating multiple instances of similar EPs or custom providers.
• Avoid version range overlaps when calling KernelRegistry::Register by using explicit SinceVersion(start, end) boundaries.
• Register custom kernels separately using KernelRegistryManager::RegisterKernelRegistry to inject implementations without touching built-in registries.
• Use single-instance patterns for EP creation, or derive custom classes with distinct type strings for multiple device configurations.
• Guard plugin registration with static initialization or std::call_once to prevent duplicate kernel insertion.

Frequently Asked Questions

What causes the "found duplicated provider" error in ONNX Runtime?

This error occurs in KernelRegistryManager::RegisterKernels (lines 33-48 of kernel_registry_manager.cc) when two execution provider instances return the same string from IExecutionProvider::Type(). The manager maintains a map keyed by provider type strings and rejects duplicate keys. Create only one instance per EP type, or override Type() to return unique identifiers for distinct configurations.

How do I register kernels for multiple GPU devices without conflicts?

Derive separate execution provider classes from the base CUDA provider and override the Type() method to return distinct strings (e.g., "CUDA0" and "CUDA1"). According to the microsoft/onnxruntime source code, the KernelRegistryManager stores registries in a map keyed by these type strings, so unique identifiers eliminate duplicate provider conflicts while allowing multiple CUDA EPs in one session.

Why does my custom kernel conflict with built-in implementations even with different provider types?

If you receive the error "Failed to add kernel for ... Conflicting with a registered kernel," the conflict occurs within a single KernelRegistry instance, not across the manager. Check that your kernel's version range (defined via KernelDefBuilder::SinceVersion) does not overlap with existing registrations in that specific registry. Alternatively, register your kernels in a separate registry using RegisterKernelRegistry, which is searched before built-in registries according to SearchKernelRegistry in kernel_registry_manager.cc.

Can I replace a built-in kernel with my custom implementation?

Yes, but you must avoid version conflicts in the same registry. The cleanest approach is to create a custom KernelRegistry containing your replacement kernels and register it via KernelRegistryManager::RegisterKernelRegistry. Because the manager searches custom registries before built-in ones (around line 84 of kernel_registry_manager.cc), your implementation will be selected first without needing to unregister the original.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}