Hardware Platforms Supported by LiteRT: CPU, GPU, and NPU Coverage

LiteRT delivers cross-platform on-device ML inference across Android, iOS, Linux, macOS, Windows, Web, and IoT devices, supporting CPU fallback on all platforms, GPU acceleration via OpenCL/OpenGL (Android), Metal (Apple), and WebGPU (desktop/Web), plus vendor-specific NPUs including Google Tensor, Qualcomm, MediaTek, Samsung, Intel, and Broadcom.

The google-ai-edge/LiteRT repository provides a unified machine learning runtime engineered to abstract hardware differences across diverse compute environments. As a cross-platform on-device ML solution, LiteRT automatically discovers and leverages available CPUs, GPUs, and NPUs through a dynamic plugin architecture. This article examines the specific hardware platforms supported by LiteRT, detailing how the runtime detects accelerators and delegates execution to optimal hardware backends.

CPU, GPU, and NPU Architecture Overview

LiteRT implements a hardware abstraction layer that unifies access to diverse compute units through a single C API. According to litert/c/internal/litert_accelerator.h, the runtime discovers available hardware via LiteRtGetNumAccelerators and LiteRtGetAccelerator, which enumerate all registered accelerators including CPU, GPU, and NPU backends.

The architecture guarantees CPU-only fallback on every supported platform, ensuring model execution succeeds even when specialized accelerators are unavailable. For GPU acceleration, LiteRT employs platform-specific delegates: OpenCL and OpenGL on Android, Metal on iOS and macOS, and WebGPU on Linux, macOS, Windows, and Web environments.

Supported Hardware Platforms and Acceleration Matrix

The official platform support matrix in README.md (lines 51-64) documents hardware compatibility across target operating systems:

• Android: CPU (always available), GPU via OpenCL/OpenGL, NPU via Google Tensor, Qualcomm, MediaTek, S.LSI, and Intel
• iOS: CPU, GPU via Metal, NPU via Apple Neural Engine (ANE)
• Linux: CPU, GPU via WebGPU (NPU support not yet available)
• macOS: CPU, GPU via WebGPU and Metal, NPU via ANE
• Windows: CPU, GPU via WebGPU, Intel NPU (coming soon)
• Web: CPU, GPU via WebGPU (NPU support coming soon)
• IoT: CPU, GPU via WebGPU, plus Broadcom and Raspberry Pi NPUs (coming soon)

Dynamic Accelerator Registration

Vendor-specific hardware support is implemented through a plugin registration system defined in litert/runtime/accelerators/auto_registration.cc. During initialization, each vendor plugin calls LiteRtCreateAccelerator to register its capabilities with the runtime.

For NPU support, plugins must implement hardware query interfaces including LiteRtGetCompilerPluginSupportedHardware, LiteRtGetNumCompilerPluginSupportedSocModels, and LiteRtGetCompilerPluginSupportedSocModel. Examples of these implementations appear in litert/vendors/qualcomm/qnn_compiler_plugin.cc for Qualcomm hardware and litert/vendors/mediatek/compiler/compiler_plugin.cc for MediaTek platforms.

Runtime Hardware Detection

LiteRT detects specific System-on-Chip (SoC) models at runtime to determine NPU compatibility. On Android, the Kotlin implementation in litert/kotlin/src/main/kotlin/com/google/ai/edge/litert/AcceleratorProvider.kt (lines 31-42 and 55-66) checks Build.SOC_MANUFACTURER and Build.SOC_MODEL against known supported hardware lists for Qualcomm, MediaTek, and Google Tensor.

val npuSupported = NpuCompatibilityChecker.Default.isDeviceSupported()
// Returns true only if the device matches a known Qualcomm, MediaTek, or Google Tensor SOC

This compatibility layer ensures that NPU acceleration is invoked only on validated hardware configurations, preventing execution errors on unsupported devices.

Enumerating and Selecting Accelerators

Applications can query available hardware programmatically using the C API defined in litert/c/internal/litert_accelerator.h:

#include "litert/c/litert.h"

LiteRtEnvironment env;
LiteRtCreateEnvironment(&env);

size_t num_accelerators = 0;
LiteRtGetNumAccelerators(env, &num_accelerators);

for (size_t i = 0; i < num_accelerators; ++i) {
  LiteRtAccelerator acc;
  LiteRtGetAccelerator(env, i, &acc);
  // Query accelerator properties...
}
LiteRtDestroyEnvironment(env);

The runtime automatically selects the optimal accelerator during context creation. As implemented in litert/runtime/accelerators/dispatch/dispatch_accelerator.cc (line 147), LiteRtCreateExecutionContext evaluates registered accelerators and delegates execution to the highest-performance available hardware without requiring manual configuration.

#include "litert/litert.h"

LiteRtModel model;
LiteRtCreateModelFromPath("my_model.tflite", &model);

LiteRtExecutionContext exec_ctx;
LiteRtCreateExecutionContext(model, &exec_ctx);

// Automatic accelerator selection occurs here
LiteRtInvoke(exec_ctx);

Summary

• LiteRT supports CPU inference universally across Android, iOS, Linux, macOS, Windows, Web, and IoT platforms through the google-ai-edge/LiteRT runtime.
• GPU acceleration varies by platform: OpenCL/OpenGL on Android, Metal on iOS/macOS, and WebGPU on Linux/macOS/Windows/Web.
• NPU support is vendor-specific, covering Google Tensor, Qualcomm, MediaTek, Samsung, Intel, Broadcom, and Raspberry Pi hardware via dynamic plugins.
• Hardware detection relies on SoC model checking in AcceleratorProvider.kt and runtime enumeration via LiteRtGetNumAccelerators in litert_accelerator.h.
• Automatic selection eliminates manual delegate configuration by evaluating registered accelerators during LiteRtCreateExecutionContext.

Frequently Asked Questions

Does LiteRT support GPU acceleration on all platforms?

Yes. LiteRT provides GPU delegates for every supported operating system. Android devices use OpenCL or OpenGL, Apple platforms use Metal, and desktop/Web environments use WebGPU. The runtime automatically detects compatible GPUs and selects the appropriate backend without manual intervention.

Which NPUs are currently supported by LiteRT?

LiteRT supports Google Tensor, Qualcomm (Snapdragon), MediaTek, Samsung (S.LSI), Intel, Broadcom, and Raspberry Pi NPUs on Android and IoT platforms, plus the Apple Neural Engine (ANE) on iOS and macOS. Additional Intel NPU support for Windows and Web NPU support are marked as coming soon in the platform matrix.

How does LiteRT detect available hardware at runtime?

The runtime enumerates accelerators through the C API functions LiteRtGetNumAccelerators and LiteRtGetAccelerator defined in litert/c/internal/litert_accelerator.h. On Android, NPU compatibility is verified by checking the device SoC manufacturer and model against known supported lists in AcceleratorProvider.kt.

Can LiteRT run on devices without GPU or NPU hardware?

Yes. LiteRT guarantees CPU-only fallback on all supported platforms. If no GPU or NPU is available or compatible, the runtime automatically executes models on the CPU, ensuring consistent functionality across all device tiers.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}