# LiteRT Ecosystem Tools: Complete Guide to Command-Line Utilities for Model Deployment > Explore the LiteRT ecosystem and its 13 command-line tools for deploying, analyzing, and debugging TensorFlow Lite models on CPU, GPU, and NPU hardware. Get started with google-ai-edge/LiteRT today. - Repository: [google-ai-edge/LiteRT](https://github.com/google-ai-edge/LiteRT) - Tags: deep-dive - Published: 2026-03-13 --- **The LiteRT ecosystem includes 13 command-line tools for running, benchmarking, analyzing, and debugging TensorFlow Lite models across CPU, GPU, and NPU hardware, all located in the `litert/tools/` directory of the google-ai-edge/LiteRT repository.** The LiteRT (Lite Runtime) project provides a self-contained suite of utilities that enable developers to deploy and validate machine learning models on diverse hardware backends. These tools share a unified architecture based on the `ai_edge_litert::CompiledModel` C++ API and support hardware-specific optimizations through compiler plugins and dispatch libraries. Whether you need to benchmark latency on an NPU or isolate a numerically unstable operator, the LiteRT ecosystem offers specialized binaries for every stage of the model deployment pipeline. ## Model Execution and Benchmarking Tools The execution tools form the foundation of the LiteRT ecosystem, providing both full-featured and minimal interfaces for model inference. ### run_model **`run_model`** is the primary full-featured runner located at `litert/tools/run_model.cc`. It supports all hardware accelerators, handles input directories, prints tensor values, performs numerical comparisons against CPU baselines, and loads compiler plugins for target-specific optimizations. The tool parses command-line options through the `LitertToolOptions` struct defined in [`litert/tools/flags/flags.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/tools/flags/flags.h), then initializes a `CompiledModel` instance. When `--accelerator=gpu` or `--accelerator=npu` is specified, the tool automatically loads the appropriate dispatch library from `litert/vendors/` using the `DispatchLibraryLoader` class. ### run_model_simple For quick sanity checks, **`run_model_simple`** (`litert/tools/run_model_simple.cc`) provides a minimal implementation that loads a `.tflite` file and performs a single inference without additional diagnostics. This tool is ideal for verifying that a model loads correctly on a new hardware target before running comprehensive benchmarks. ### benchmark_litert_model_main Performance analysis relies on **`benchmark_litert_model_main`** (`litert/tools/benchmark_litert_model_main.cc`). This tool executes a model for a configurable number of iterations, reporting detailed latency statistics, throughput metrics, and memory usage. It supports warmup runs to stabilize cache states and can load compiler plugins via the `--compiler_plugin_library_dir` flag to measure optimized graph performance. ## Model Analysis and Inspection Tools Before deployment, understanding model structure and metadata is critical. The LiteRT ecosystem provides several analysis utilities. ### analyze_model_main **`analyze_model_main`** (`litert/tools/analyze_model_main.cc`) generates high-level model summaries or detailed operator lists, including subgraph structures, tensor shapes, and operation codes. This tool helps developers verify that model conversion preserved the expected architecture. ### dump_ops_main For debugging specific operators, **`dump_ops_main`** (`litert/tools/dump_ops_main.cc`) splits a model into individual-operator sub-models. It supports batch processing and opcode filtering, enabling isolated testing of specific layers that may cause issues on particular hardware. ### metadata_util_main **`metadata_util_main`** (`litert/tools/metadata_util_main.cc`) inspects or extracts model metadata, including custom operator registrations and version information. This is essential when working with models that contain custom operations or specific runtime requirements. ### extract_bytecode The **`extract_bytecode`** tool (`litert/tools/extract_bytecode.cc`) extracts compiled bytecode from LiteRT models for offline inspection. This allows developers to analyze the low-level instructions generated for specific accelerators without executing the model. ## Hardware Validation and Debugging Tools Ensuring numerical correctness across different hardware backends requires specialized validation tools. ### npu_numerics_check and gpu_numerics_check **`npu_numerics_check`** (`litert/tools/npu_numerics_check.cc`) and **`gpu_numerics_check`** (`litert/tools/gpu_numerics_check.cc`) execute models on both CPU and their respective accelerators (NPU or GPU), then compare output tensors numerically. These tools report maximum absolute and relative errors, helping identify precision loss introduced by hardware-specific quantization or operator implementations. ### culprit_finder_main When a model crashes or produces incorrect results, **`culprit_finder_main`** (`litert/tools/culprit_finder/culprit_finder_main.cc`) performs a binary search through the operator list to isolate the specific operation causing the failure. The tool progressively creates sub-models containing smaller slices of the original graph until it identifies the offending operator's index, opcode, and description via the `OperatorRegistry`. ### accuracy_debugger_main **`accuracy_debugger_main`** (`litert/tools/accuracy_debugger/accuracy_debugger_main.cc`) runs models while logging per-operator accuracy statistics. This granular visibility helps pinpoint where precision loss occurs during dispatch to hardware accelerators, particularly useful when debugging quantized models. ## Compiler Plugin and Optimization Tools LiteRT supports ahead-of-time optimizations through compiler plugins that perform graph transformations for specific SoCs. ### apply_plugin_main **`apply_plugin_main`** (`litert/tools/apply_plugin_main.cc`) applies hardware-specific compiler plugins to a model, producing an optimized output file. The tool loads plugins via `PluginLoader` (implemented in `litert/plugins/plugin_loader.cc`) and invokes `Plugin::Run` to perform target-specific optimizations such as operator fusion or quantization before model execution. ### intel_openvino_flags_example For vendor-specific implementations, **`intel_openvino_flags_example`** (`litert/tools/flags/vendors/intel_openvino_flags_example.cc`) demonstrates how to define and parse backend-specific command-line flags. This serves as a reference for integrating new hardware vendors into the LiteRT ecosystem. ## Shared Architecture and Implementation Details All LiteRT ecosystem tools share common architectural patterns that ensure consistency across the suite. ### Unified Flag Parsing Every tool includes [`litert/tools/flags/flags.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/tools/flags/flags.h), which defines the `LitertToolOptions` struct containing shared command-line options. Flags parse through `absl::ParseCommandLine`, providing standardized interfaces for `--graph`, `--accelerator`, `--input_dir`, and diagnostic flags like `--print_tensors` and `--compare_numerical`. ### CompiledModel API Integration At the core of each tool lies the `ai_edge_litert::CompiledModel` C++ API. This interface handles `.tflite` file loading, input/output tensor binding, and inference invocation. The API abstracts hardware differences, allowing the same tool binary to target CPU, GPU, or NPU through configuration flags. ### Dispatch Library Loading When targeting specialized hardware, tools construct a `DispatchLibraryLoader` that dynamically opens vendor libraries (e.g., `libLiteRtDispatch_Qualcomm.so` or `libLiteRtDispatch_MediaTek.so`) using `dlopen`. The loader resolves the `CreateDispatch` entry point required by `CompiledModel` to forward operations to the appropriate NPU or GPU driver. ### Tensor I/O and Diagnostics Input tensors can be supplied as raw binary files in a directory specified by `--input_dir`. The `LoadInputsFromDirectory` helper (declared in [`litert/tools/util.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/tools/util.h)) maps files to signature input names and performs type-aware casting. When `--print_tensors` is enabled, `PrintTensorValues` outputs tensor contents with configurable sample sizes via `--sample_size`. ## Practical Usage Examples ### Running a Model on GPU with Tensor Inspection Build and execute the full-featured runner to validate GPU execution: ```bash bazel build //litert/tools:run_model ./bazel-bin/litert/tools/run_model \ --graph=/tmp/my_model.tflite \ --accelerator=gpu \ --print_tensors=true \ --sample_size=5 ``` Under the hood, `run_model.cc` instantiates `CompiledModel`, selects the GPU dispatch library, runs inference, and calls `PrintTensorValues` with the specified sample size. ### Benchmarking with NPU Acceleration and Compiler Plugins Measure performance on Qualcomm NPU hardware with graph optimizations: ```bash bazel build //litert/tools:benchmark_litert_model_main ./bazel-bin/litert/tools/benchmark_litert_model_main \ --graph=/tmp/vision_model.tflite \ --accelerator=npu \ --dispatch_library_dir=/opt/qnn_dispatch \ --compiler_plugin_library_dir=/opt/qnn_plugin \ --num_runs=100 \ --warmup_runs=5 ``` The tool loads the NPU dispatch library, pre-loads the compiler plugin via `PluginLoader` for target-specific optimizations, and reports standardized latency and throughput statistics. ### Isolating Crashing Operators Use the culprit finder to identify unstable operations: ```bash bazel build //litert/tools/culprit_finder:culprit_finder_main ./bazel-bin/litert/tools/culprit_finder/culprit_finder_main \ --graph=/tmp/faulty_model.tflite \ --accelerator=npu \ --dispatch_library_dir=/opt/qnn_dispatch ``` The binary performs binary search isolation on the operator list, printing the index and opcode of the first operation that triggers a crash or numerical discrepancy. ## Summary - The LiteRT ecosystem provides **13 specialized command-line tools** for complete model lifecycle management, from execution to debugging. - All tools reside in `litert/tools/` and share common infrastructure through `LitertToolOptions` and the `CompiledModel` API. - **Execution tools** like `run_model` and `benchmark_litert_model_main` support CPU, GPU, and NPU backends through the dispatch library abstraction. - **Validation tools** including `npu_numerics_check` and `culprit_finder_main` ensure numerical correctness and isolate hardware-specific failures. - **Compiler plugin tools** such as `apply_plugin_main` enable ahead-of-time optimizations for specific SoC targets. - The architecture supports **modular builds** while maintaining consistent CLI patterns across all utilities. ## Frequently Asked Questions ### What is the difference between run_model and run_model_simple? **`run_model`** is the full-featured runner supporting all accelerators, input directories, tensor printing, numerical comparison, and compiler plugin loading, while **`run_model_simple`** provides a minimal implementation that only loads a model and performs a single inference. Use `run_model_simple` for quick sanity checks when you don't need diagnostic features or hardware acceleration configuration. ### How do I validate numerical accuracy between CPU and NPU execution? Use the **`npu_numerics_check`** tool located at `litert/tools/npu_numerics_check.cc`. This utility runs your model on both CPU and NPU, compares output tensors using max absolute and relative error metrics, and reports discrepancies. For GPU validation, use the equivalent **`gpu_numerics_check`** tool instead. ### Where are the shared command-line flags defined in the LiteRT ecosystem? All tools share common flag definitions in **[`litert/tools/flags/flags.h`](https://github.com/google-ai-edge/LiteRT/blob/main/litert/tools/flags/flags.h)**, which declares the `LitertToolOptions` struct. Each tool parses these flags using `absl::ParseCommandLine`, ensuring consistent options for model paths, accelerator selection, input directories, and diagnostic settings across the entire tool suite. ### How do I apply hardware-specific optimizations to my model before deployment? Use **`apply_plugin_main`** (`litert/tools/apply_plugin_main.cc`) with the `--compiler_plugin_library_dir` flag pointing to your vendor-specific plugin. The tool loads the plugin via `PluginLoader` and invokes `Plugin::Run` to perform ahead-of-time graph transformations, outputting an optimized model file ready for deployment on the target hardware.