# What Types of AI Models Can LiteRT Run? A Complete Guide to On-Device AI

> Explore the AI models LiteRT supports. Run TensorFlow Lite models for vision, NLP, audio, generative AI and more on optimized hardware accelerators like CPU, GPU, NPU.

- Repository: [google-ai-edge/LiteRT](https://github.com/google-ai-edge/LiteRT)
- Tags: deep-dive
- Published: 2026-03-13

---

**LiteRT can run any TensorFlow Lite (.tflite) model, including vision classifiers, natural language processors, audio detectors, and generative AI models like LLMs and diffusion models, while automatically selecting the optimal CPU, GPU, or NPU accelerator.**

The google-ai-edge/LiteRT repository provides a high-performance runtime for deploying AI models directly on edge devices. Whether you need to run image classification, sentiment analysis, or large language model inference, LiteRT executes these workloads efficiently by leveraging hardware-specific delegates. This guide explains exactly what types of AI models LiteRT supports and how it processes them at the source code level.

## Supported AI Model Categories in LiteRT

### Vision Models

LiteRT provides first-class APIs for computer vision tasks through the Task Library. As documented in [`tflite/g3doc/inference_with_metadata/task_library/overview.md`](https://github.com/google-ai-edge/LiteRT/blob/main/tflite/g3doc/inference_with_metadata/task_library/overview.md), supported vision APIs include **ImageClassifier**, **ObjectDetector**, **ImageSegmenter**, **ImageEmbedder**, and **ImageSearcher**. These APIs handle models ranging from MobileNet-based classifiers to complex segmentation networks, enabling use cases like object detection, visual search, and feature extraction.

### Natural Language Models

For text processing, LiteRT supports **NLClassifier**, **BertNLClassifier**, **BertQuestionAnswerer**, **TextSearcher**, and **TextEmbedder**. These task-specific wrappers handle sentiment analysis, intent classification, question answering, and semantic search models. The Bert-based implementations allow running transformer models optimized for on-device constraints.

### Audio Models

LiteRT supports audio classification through the **AudioClassifier** API, enabling keyword spotting and sound event detection models. This covers models trained to identify specific audio patterns or speech commands in real-time on device.

### Generative AI Models

Through **LiteRT-LM**, the runtime supports large language models (LLMs) and diffusion models for on-device text generation and image generation. This extends LiteRT beyond traditional discriminative models into generative AI workloads, allowing models like Gemma 2B to run locally.

### Custom and Flex Delegate Models

LiteRT can execute any `.tflite` model, including those with custom TensorFlow operations, via the **Flex delegate**. This allows models using unsupported ops to run on-device by falling back to TensorFlow kernels when native TFLite kernels aren't available.

## How LiteRT Executes Different Model Types

The execution pipeline in LiteRT follows a structured dispatch system that adapts to model requirements.

**Model Loading**: The runtime parses the flat-buffer format using `FlatBufferModel` and constructs a graph of `TfLiteNode` objects. This occurs in the core compilation phase handled by [`litert/cc/litert_compiled_model.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/cc/litert_compiled_model.h).

**Operator Dispatch**: Each node routes to either a built-in kernel, a hardware delegate (GPU, NPU), or the Flex delegate. The decision logic resides in functions like `IsNodeSupportedByDelegate`, with platform-specific implementations found in files like [`tflite/delegates/coreml/README.md`](https://github.com/google-ai-edge/LiteRT/blob/main/tflite/delegates/coreml/README.md) for Core ML support.

**Execution**: The `CompiledModel::Run` method executes the graph on the selected backend. According to `cmake_example/run_model_simple.cc`, this process handles tensor allocation through `TensorBuffer` abstractions that support zero-copy GPU buffers when available.

**Task-Specific Helpers**: High-level wrappers in C++, Java, and Swift (such as `ImageClassifier`) abstract the low-level details. These helpers, documented in the Task Library overview, provide simple inference flows while internally managing the complex delegate selection.

## Code Examples for Running AI Models in LiteRT

### Running a Generic .tflite Model (C++)

The simplest way to run any TensorFlow Lite model uses the minimal example in `cmake_example/run_model_simple.cc`:

```cpp
// Compile and run any .tflite model, choosing hardware automatically.
int main(int argc, char** argv) {
  absl::ParseCommandLine(argc, argv);
  // Pass the model path with --graph=your_model.tflite
  // Choose accelerators with --accelerator=cpu,gpu,npu
  auto status = litert::RunModel();
  return status ? EXIT_SUCCESS : EXIT_FAILURE;
}

```

### Vision Task Library Implementation (C++)

For structured vision tasks, use the Task Library API:

```cpp
// High-level API: only a few lines needed.
auto options = ImageClassifierOptions::Create();
options->SetModelFile("mobilenet_v2.tflite");
auto classifier = ImageClassifier::Create(options).value();

ImageData image = LoadImage("cat.jpg");
auto result = classifier->Classify(image).value();
for (const auto& label : result.classifications()) {
  std::cout << label.id << ": " << label.score << "\n";
}

```

### Large Language Model Inference (Python)

LiteRT-LM provides Python bindings for generative models:

```python

# LiteRT-LM provides a simple Python wrapper.

from litert_lm import LLM
model = LLM("gemma_2b.tflite")
output = model.generate("Explain quantum computing in one sentence.")
print(output)

```

## Key Source Files for Model Support

Understanding the repository structure helps clarify LiteRT's model handling capabilities:

- **[`litert/cc/litert_compiled_model.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/cc/litert_compiled_model.h)**: Defines the core `CompiledModel` class for loading `.tflite` files and preparing them for execution.
- **[`litert/cc/litert_environment.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/cc/litert_environment.h)**: Manages runtime resources including thread pools and hardware backend initialization.
- **[`litert/cc/litert_options.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/cc/litert_options.h)**: Configures hardware accelerators, memory budgets, and execution flags for different model types.
- **[`tflite/g3doc/inference_with_metadata/task_library/overview.md`](https://github.com/google-ai-edge/LiteRT/blob/main/tflite/g3doc/inference_with_metadata/task_library/overview.md)**: Documents all supported task-specific APIs and their corresponding model families.
- **`cmake_example/run_model_simple.cc`**: Demonstrates minimal end-to-end execution of any `.tflite` model with automatic accelerator selection.
- **[`tflite/delegates/coreml/README.md`](https://github.com/google-ai-edge/LiteRT/blob/main/tflite/delegates/coreml/README.md)**: Illustrates how LiteRT maps operators to platform-specific delegates like Core ML, GPU, and NPU.

## Summary

- LiteRT executes any **TensorFlow Lite (.tflite)** model, including quantized, float, and control-flow variants.
- First-class APIs support **vision** (classification, detection, segmentation), **NLP** (BERT, text classification), and **audio** (keyword spotting) tasks.
- **LiteRT-LM** enables on-device execution of generative AI models including large language models and diffusion models.
- The **Flex delegate** extends support to custom TensorFlow operations not natively implemented in TFLite.
- Hardware acceleration is automatic, with delegates for CPU, GPU (OpenCL/OpenGL), and vendor-specific NPUs (Qualcomm, MediaTek).

## Frequently Asked Questions

### Can LiteRT run PyTorch or ONNX models directly?

No, LiteRT specifically executes **TensorFlow Lite (.tflite)** models. To run PyTorch or ONNX models, you must first convert them to the TFLite format using conversion tools like ONNX-TFLite or PyTorch's mobile exporter, then load the resulting `.tflite` file via [`litert/cc/litert_compiled_model.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/cc/litert_compiled_model.h).

### Does LiteRT support quantized models?

Yes, LiteRT fully supports quantized models including INT8 and post-training quantization. The `CompiledModel` API in [`litert_compiled_model.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert_compiled_model.h) handles quantized tensors automatically, and hardware delegates like GPU and NPU often provide optimized paths for quantized inference.

### How do I run a custom TensorFlow operation with LiteRT?

Use the **Flex delegate** by including the custom op in your model and ensuring the Flex delegate is linked. The runtime checks `IsNodeSupportedByDelegate` to determine if an op requires Flex fallback, allowing unsupported ops to execute via TensorFlow kernels on-device.

### What hardware accelerators does LiteRT support?

LiteRT automatically selects from **CPU**, **GPU** (via OpenCL or OpenGL), and **NPU** backends including Qualcomm QNN and MediaTek Neuron. The selection logic in `cmake_example/run_model_simple.cc` and related delegate files evaluates the model's operator set against each accelerator's capabilities to optimize performance.