How to Debug Applications Using LiteRT: A Complete Guide

LiteRT provides command-line tools like run_model, dump_ops, and culprit_finder alongside the LiteRtProfilerT API to inspect tensor values, isolate failing operators, and profile execution on CPU, GPU, and NPU accelerators.

Debugging applications using LiteRT (Lite Runtime) requires a systematic approach to inspecting model execution, isolating problematic operators, and validating performance across hardware accelerators. The google-ai-edge/LiteRT repository ships with a comprehensive toolkit located under litert/tools/ that enables developers to diagnose numerical errors, crashes, and latency issues without modifying application source code.

Command-Line Debugging Tools

The litert/tools/ directory contains several standalone utilities built via Bazel that support interactive debugging workflows.

Inspect Tensor Values with run_model

The run_model tool loads a .tflite model via the CompiledModel API and executes it on your selected accelerator while providing visibility into intermediate tensor values.

Use the --print_tensors=true flag to dump a configurable sample of output tensors, or --compare_numerical=true to fill inputs with a rotating pattern and report per-tensor statistics. This is the fastest way to verify that your model produces expected outputs on specific hardware.

# Basic CPU execution with tensor inspection

run_model --graph=model.tflite --print_tensors=true --sample_size=10

# GPU execution with numerical comparison

run_model \
  --graph=model.tflite \
  --accelerator=gpu \
  --compare_numerical=true

Reference: litert/tools/README.md contains the complete flag specification for run_model.

Isolate Operators with dump_ops

When you suspect a specific operator is misbehaving on a delegate (e.g., GPU or NPU), use dump_ops to split the model into single-operator sub-models. This creates a new TinyLiteRT model for every operator, allowing you to test individual operations in isolation.

dump_ops \
  --model_path=model.tflite \
  --output_dir=/tmp/single_ops \
  --filter_opcode=CONV_2D

Each generated model contains exactly one operator, making it trivial to identify which specific op produces NaN values or crashes when executed with run_model.

Automated Binary Search with culprit_finder

For models with many operators, manually testing each one is inefficient. The culprit_finder tool automates the search for offending operators using either linear or binary search strategies when a delegate produces NaNs or large numeric errors.

Build the tool with debug delegate support using the -DTFLITE_DEBUG_DELEGATE flag:

bazel build -c opt --config=android_arm64 \
  litert/tools/culprit_finder:culprit_finder_main \
  --copt=-DTFLITE_DEBUG_DELEGATE

Execute with binary search to rapidly isolate the problematic operator range:

adb push bazel-bin/litert/tools/culprit_finder/culprit_finder_main /data/local/tmp
adb shell /data/local/tmp/culprit_finder_main \
  --graph=/data/local/tmp/model.tflite \
  --use_gpu=true \
  --find_nan=true \
  --find_numeric_error=true \
  --search_strategy=binary

Key flags include --min_numeric_error (default 0.0001) to define error thresholds and --search_strategy to toggle between linear and binary approaches.

Performance Baselines with benchmark_model

Before debugging functional issues, establish a performance baseline using benchmark_model. This tool runs configurable iterations while reporting latency, memory consumption, and throughput, helping you distinguish between correctness bugs and performance regressions.

benchmark_model \
  --graph=model.tflite \
  --use_gpu \
  --num_runs=100 \
  --warmup_runs=5

Programmatic Profiling with LiteRtProfilerT

For debugging inside your application code, LiteRT exposes the LiteRtProfilerT class defined in litert/runtime/profiler.h. This lightweight C++ profiler collects per-op execution timing, memory usage, and custom user events without external tooling.

Attach the profiler to your CompiledModel before invocation:

#include "litert/runtime/profiler.h"

LiteRtProfilerT profiler;
CompiledModel model = CompiledModel::FromFile("model.tflite");

model.SetProfiler(&profiler);
model.Invoke(inputs, outputs);
profiler.DumpToStdout();

For Python applications using the LiteRT bindings, instantiate the Profiler class and attach it similarly:

from ai_edge_litert import CompiledModel, Profiler

prof = Profiler()
model = CompiledModel.from_file("model.tflite")
model.set_profiler(prof)
model.invoke(input_dict)
prof.dump()

Debug Flags and Environment Variables

LiteRT respects several TensorFlow Lite-style debug flags that can be enabled at runtime or compile time.

Build-time flags:

• -DTFLITE_DEBUG_DELEGATE: Enables delegate debugging features required by culprit_finder to restrict which nodes are delegated via first_delegate_node_index and last_delegate_node_index flags.

Runtime environment variables:

• TFLITE_LOGGING=1: Emits verbose runtime logs useful for tracing execution flow.
• debug.tflite.trace (Android system property): Enables low-level kernel tracing.

Enable Android tracing before execution:

adb shell setprop debug.tflite.trace 1
run_model --graph=model.tflite --accelerator=gpu
adb shell setprop debug.tflite.trace 0

Diagnosing Quantization Errors

When debugging quantization-induced accuracy loss, use the Quantization Debugger from tflite/tools/optimize/debugging/. This generates a debug model containing extra ops that compute per-layer error metrics like mean absolute error.

from tensorflow.lite.tools.optimize.debugging.python import debugger

quant_debugger = debugger.QuantizationDebugger(
    quant_debug_model_path="model_debug.tflite",
    debug_dataset=my_dataset)

quant_debugger.run()
print(quant_debugger.layer_statistics)

The resulting .tflite debug model can be executed with LiteRT tools like run_model to verify that the LiteRT runtime reproduces the observed quantization errors.

Summary

• Use run_model with --print_tensors to verify numerical correctness across CPU, GPU, and NPU accelerators.
• Isolate specific operators using dump_ops when you suspect individual op failures.
• Automate operator search with culprit_finder and binary search (--search_strategy=binary) to rapidly locate NaN-producing or high-error ops.
• Profile application performance programmatically using LiteRtProfilerT in C++ or the Profiler class in Python.
• Enable -DTFLITE_DEBUG_DELEGATE at compile time and use debug.tflite.trace on Android for low-level execution tracing.
• Diagnose quantization issues using the TensorFlow Lite Quantization Debugger tools located in tflite/tools/optimize/debugging/.

Frequently Asked Questions

How do I find which operator is causing NaN values in LiteRT?

Use the culprit_finder tool with the --find_nan=true flag. Build it with -DTFLITE_DEBUG_DELEGATE and run it with --search_strategy=binary to automatically binary-search through the operator list and identify the minimal set of ops producing NaN values on your target accelerator.

Can I debug LiteRT models on Android devices?

Yes. All command-line tools (run_model, benchmark_model, culprit_finder) can be built for Android using Bazel configs like --config=android_arm64, pushed to /data/local/tmp via adb push, and executed via adb shell. Additionally, set the Android system property debug.tflite.trace to 1 to enable kernel-level tracing.

What is the difference between dump_ops and culprit_finder?

dump_ops statically splits a model into single-operator sub-models, useful for manual inspection of specific opcodes. culprit_finder dynamically executes the full model with varying operator subsets to automatically detect which operators cause numerical errors or NaNs, supporting both linear and binary search strategies for efficiency.

How do I profile memory usage in LiteRT applications?

Attach the LiteRtProfilerT object to your CompiledModel instance before calling Invoke. After execution, call DumpToStdout() to view per-operator memory usage and timing statistics. In Python, use the Profiler class from ai_edge_litert and call dump() after model execution to retrieve the same metrics programmatically.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}