# LiteRT Usage Examples: Complete Guide to On-Device Inference

> Explore LiteRT usage examples in the official repository including interpreter demos and CompiledModel API implementations for on-device inference on hardware accelerators.

- Repository: [google-ai-edge/LiteRT](https://github.com/google-ai-edge/LiteRT)
- Tags: tutorial
- Published: 2026-03-13

---

**You can find official LiteRT usage examples in the `cmake_example/` directory of the google-ai-edge/LiteRT repository, including minimal interpreter demos and full CompiledModel API implementations that demonstrate hardware accelerator selection and end-to-end inference.**

The LiteRT runtime (formerly TensorFlow Lite) provides self-contained examples in the `google-ai-edge/LiteRT` repository that demonstrate everything from basic model loading to advanced hardware-accelerated inference. These examples serve as the definitive reference for implementing on-device machine learning in C++ applications.

## Where to Find LiteRT Usage Examples

The repository organizes working examples into three main categories: minimal interpreter demonstrations, full runtime API implementations, and vendor extension samples.

### Minimal TensorFlow Lite Interpreter

Located at `cmake_example/tflite_minimal.cc`, this bare-bones program demonstrates direct TensorFlow Lite API usage without LiteRT abstractions. It loads a `.tflite` file, builds an interpreter using `tflite::InterpreterBuilder`, allocates tensors, and runs inference.

### Full LiteRT CompiledModel API

The `cmake_example/run_model_simple.cc` file provides a complete end-to-end example using the modern LiteRT runtime. It creates a `litert::Environment`, selects hardware accelerators (CPU/GPU/NPU), builds a `litert::CompiledModel`, manages input/output buffers, and executes the model.

### Vendor Extension Examples

For custom backend development, `litert/vendors/examples/example_transformations.cc` shows how to plug custom transformation passes into the LiteRT optimization pipeline. Python developers can reference [`litert/python/aot/vendors/example/example_backend.py`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/python/aot/vendors/example/example_backend.py) for Ahead-of-Time (AOT) compiler integration.

## Key LiteRT APIs Demonstrated

The examples illustrate the core LiteRT runtime workflow through specific API calls implemented in the `litert/cc/` directory.

### Environment Creation

The `litert::Environment::Create` method initializes global runtime state, loading system libraries and preparing hardware interfaces. This is implemented in [`litert/cc/litert_environment.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/cc/litert_environment.h) and `litert/cc/litert_environment.cc`.

### Hardware Accelerator Selection

The `Options` class manages hardware acceleration through `SetHardwareAccelerators`. The example parses command-line flags to configure CPU, GPU, or NPU backends via the `HwAcceleratorSet` structure, defined in [`litert/cc/litert_options.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/cc/litert_options.h).

### Model Compilation and Execution

`litert::CompiledModel::Create` compiles FlatBuffer models for selected accelerators. The `Run` method executes inference using prepared `TensorBuffer` objects for inputs and outputs, as defined in [`litert/cc/litert_compiled_model.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/cc/litert_compiled_model.h). The `CreateInputBuffers` and `CreateOutputBuffers` methods allocate type-safe memory buffers matching the model signature.

## Complete Code Examples

### Minimal TFLite Interpreter

This example from `cmake_example/tflite_minimal.cc` shows the low-level TensorFlow Lite approach:

```cpp
int main(int argc, char* argv[]) {
  if (argc != 2) {
    fprintf(stderr, "tflite_minimal <tflite model>\n");
    return 1;
  }
  const char* filename = argv[1];

  // Load the model.
  std::unique_ptr<tflite::FlatBufferModel> model =
      tflite::FlatBufferModel::BuildFromFile(filename);
  TFLITE_MINIMAL_CHECK(model != nullptr);

  // Build the interpreter.
  tflite::ops::builtin::BuiltinOpResolver resolver;
  tflite::InterpreterBuilder builder(*model, resolver);
  std::unique_ptr<tflite::Interpreter> interpreter;
  builder(&interpreter);
  TFLITE_MINIMAL_CHECK(interpreter != nullptr);

  // Allocate tensors, run, and print state.
  TFLITE_MINIMAL_CHECK(interpreter->AllocateTensors() == kTfLiteOk);
  printf("=== Pre-invoke Interpreter State ===\n");
  tflite::PrintInterpreterState(interpreter.get());

  // TODO: Fill input tensors here.

  TFLITE_MINIMAL_CHECK(interpreter->Invoke() == kTfLiteOk);
  printf("\n=== Post-invoke Interpreter State ===\n");
  tflite::PrintInterpreterState(interpreter.get());

  // TODO: Read output tensors here.
  return 0;
}

```

### LiteRT CompiledModel API

This example from `cmake_example/run_model_simple.cc` demonstrates the modern LiteRT runtime:

```cpp
int main(int argc, char** argv) {
  absl::ParseCommandLine(argc, argv);

  // 1️⃣ Create runtime environment.
  LITERT_ASSIGN_OR_RETURN(auto env, litert::Environment::Create({}));

  // 2️⃣ Choose hardware (cpu,gpu,npu) via command-line flag.
  LITERT_ASSIGN_OR_RETURN(auto options, litert::Options::Create());
  options.SetHardwareAccelerators(litert::GetAccelerator());

  // 3️⃣ Load and compile the model.
  LITERT_ASSIGN_OR_RETURN(
      auto compiled_model,
      litert::CompiledModel::Create(env,
                                  absl::GetFlag(FLAGS_graph),  // path to .tflite
                                  options));

  // 4️⃣ Prepare buffers for the specified signature.
  size_t sig = absl::GetFlag(FLAGS_signature_index);
  LITERT_ASSIGN_OR_RETURN(auto input_buffers,
                          compiled_model.CreateInputBuffers(sig));
  // Optionally fill with dummy data for numerical checks.
  for (auto& buf : input_buffers) { LITERT_RETURN_IF_ERROR(FillInputBuffer(buf)); }

  LITERT_ASSIGN_OR_RETURN(auto output_buffers,
                          compiled_model.CreateOutputBuffers(sig));

  // 5️⃣ Run inference.
  litert::Expected<void> status =
      compiled_model.Run(sig, input_buffers, output_buffers);
  if (!status) { ABSL_LOG(ERROR) << status.Error().Message(); return EXIT_FAILURE; }

  ABSL_LOG(INFO) << "Model run completed";
  return EXIT_SUCCESS;
}

```

## Essential Source Files for LiteRT Development

When building applications based on these examples, reference these core header files:

- **[`litert/cc/litert_environment.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/cc/litert_environment.h)** – Environment creation and global runtime state management.
- **[`litert/cc/litert_options.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/cc/litert_options.h)** – Configuration container including `SetHardwareAccelerators`.
- **[`litert/cc/litert_compiled_model.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/cc/litert_compiled_model.h)** – `CompiledModel` class for loading, compiling, and running models.
- **[`litert/cc/litert_tensor_buffer.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/cc/litert_tensor_buffer.h)** – `TensorBuffer` abstraction for type-safe input/output memory handling.
- **[`cmake_example/CMakeLists.txt`](https://github.com/google-ai-edge/LiteRT/blob/main/cmake_example/CMakeLists.txt)** – Build configuration for compiling the examples.

## Summary

- The `cmake_example/` directory contains the primary **LiteRT usage examples**, ranging from minimal TFLite interpreters to full CompiledModel implementations.
- **`run_model_simple.cc`** demonstrates the modern LiteRT workflow: Environment → Options → CompiledModel → TensorBuffer → Run.
- **`tflite_minimal.cc`** provides a low-level reference using raw TensorFlow Lite APIs without LiteRT abstractions.
- Vendor extensions and custom backends are illustrated in `litert/vendors/examples/` and the Python AOT stubs.
- Key headers in `litert/cc/` define the APIs used for environment setup, hardware acceleration, and tensor management.

## Frequently Asked Questions

### What is the difference between the minimal TFLite example and the LiteRT CompiledModel example?

The minimal example in `tflite_minimal.cc` uses raw TensorFlow Lite APIs (`FlatBufferModel`, `InterpreterBuilder`) directly without LiteRT's abstraction layer. The `run_model_simple.cc` example uses the modern LiteRT runtime with `litert::CompiledModel`, which provides hardware accelerator selection, better memory management through `TensorBuffer`, and a more streamlined API for production deployments.

### How do I enable GPU or NPU acceleration in LiteRT examples?

Pass the `--accelerator` flag when running `run_model_simple.cc`. The example code parses this flag and calls `options.SetHardwareAccelerators()` with the appropriate `HwAcceleratorSet` value. This configuration happens in [`litert/cc/litert_options.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/cc/litert_options.h) and is applied during `CompiledModel::Create`.

### Are there Python examples available for LiteRT?

Yes, the repository includes Python AOT (Ahead-of-Time) backend examples in [`litert/python/aot/vendors/example/example_backend.py`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/python/aot/vendors/example/example_backend.py). This file demonstrates how to implement custom Python backends for the LiteRT compiler pipeline, though the primary runtime examples are in C++.

### How do I build and run these LiteRT examples?

Use the provided [`cmake_example/CMakeLists.txt`](https://github.com/google-ai-edge/LiteRT/blob/main/cmake_example/CMakeLists.txt) to build the examples with CMake. The build system compiles both `tflite_minimal.cc` and `run_model_simple.cc`. Run the compiled binaries with a `.tflite` model file as an argument, and optionally specify `--accelerator=gpu` or `--accelerator=npu` for the LiteRT example to test hardware acceleration.