# How to Configure Backend Options for LiteRT-LM: CPU, GPU, Vision, and Audio Backends

> Learn how to configure LiteRT-LM backend options for CPU, GPU, vision, and audio. Control inference engines via CLI or Python API for optimized performance.

- Repository: [google-ai-edge/LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM)
- Tags: how-to-guide
- Published: 2026-04-06

---

**LiteRT-LM configures hardware acceleration through the `Backend` enum, letting you set the main inference engine to CPU or GPU via CLI flags or Python arguments, while vision and audio encoders are controlled separately through the Python API only.**

LiteRT-LM is an inference framework for multimodal language models that requires explicit hardware configuration to optimize performance. Whether you are running inference on edge devices or packaging models for specific deployment targets, understanding how to configure backend options for LiteRT-LM ensures you leverage the right compute resources for text, vision, and audio processing.

## The Backend Enum Definition

All hardware backend specifications in LiteRT-LM rely on the `Backend` enum defined in [`python/litert_lm/interfaces.py`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/python/litert_lm/interfaces.py). This enumeration maps hardware targets to integer values used by the underlying C++ engine.

```python
class Backend(enum.Enum):
    """Hardware backends for LiteRT‑LM."""
    UNSPECIFIED = 0
    CPU = 3
    GPU = 4

```

The `UNSPECIFIED` value tells the engine to auto-select the backend, while `CPU` and `GPU` force execution to the respective hardware. When configuring backend options for LiteRT-LM, you reference these enum members in Python or their string equivalents in the CLI.

## Configuring the Inference Backend

The inference backend determines whether the core language model runs on CPU or GPU. You can configure this via command-line interface or Python API.

### CLI Configuration

The CLI provides the `--backend` (or `-b`) flag, defined in [`python/litert_lm_cli/main.py`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/python/litert_lm_cli/main.py), which accepts the strings `"cpu"` or `"gpu"`. The flag is available for both `run` and `benchmark` commands.

```bash
litert-lm run ./my_model.litertlm --backend=gpu

```

Behind the scenes, the [`model.py`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/model.py) module converts this string to the enum using the `_parse_backend` function:

```python
def _parse_backend(backend: str) -> litert_lm.Backend:
    backend_lower = backend.lower()
    if backend_lower == "gpu":
        return litert_lm.Backend.GPU
    return litert_lm.Backend.CPU

```

This function is located in [`python/litert_lm_cli/model.py`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/python/litert_lm_cli/model.py) and defaults to `Backend.CPU` if no match is found.

### Python API Configuration

When initializing the inference engine programmatically, pass the `backend` argument directly to the `Engine` constructor:

```python
import litert_lm

engine = litert_lm.Engine(
    model_path="my_model.litertlm",
    backend=litert_lm.Backend.GPU
)

```

If omitted, the Python API defaults to CPU execution.

## Setting Vision and Audio Encoder Backends

Unlike the main inference backend, vision and audio encoder backends are only configurable through the Python API. The `AbstractEngine` class in [`python/litert_lm/interfaces.py`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/python/litert_lm/interfaces.py) exposes these as optional dataclass fields:

```python
vision_backend: Backend | None = None
audio_backend: Backend | None = None

```

This design allows you to offload specific encoders to different hardware than the main model. For example, you might run the text model on CPU while processing vision tensors on GPU to balance memory and latency.

To configure these backends, pass the enum values when creating the `Engine`:

```python
engine = litert_lm.Engine(
    model_path="multimodal_model.litertlm",
    backend=litert_lm.Backend.CPU,          # Inference on CPU

    vision_backend=litert_lm.Backend.GPU,     # Image encoder on GPU

    audio_backend=litert_lm.Backend.CPU      # Audio encoder on CPU

)

```

If you omit `vision_backend` or `audio_backend`, they default to `Backend.UNSPECIFIED`, allowing the engine to auto-select the appropriate hardware.

## Enforcing Backend Constraints in Model Packaging

When building LiteRT-LM model files (`.litertlm`), you can embed backend constraints that restrict which hardware the model is allowed to run on. This is validated during model construction in [`schema/py/litertlm_builder.py`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/schema/py/litertlm_builder.py) via the `_validate_backend_constraints` function.

```python
def _validate_backend_constraints(backend_constraint: str) -> None:
    backends = [b.strip().lower() for b in backend_constraint.split(",")]
    valid_backends = set(Backend)
    for backend in backends:
        if backend not in valid_backends:
            raise ValueError(...)

```

To restrict a model to GPU only during packaging:

```python
builder.add_tflite_model(
    model_path="my_model.tflite",
    backend_constraint="gpu",   # Only GPU allowed at runtime

)

```

If a user attempts to load this model with `--backend=cpu`, the runtime will reject the configuration, preventing incompatible execution paths.

## Complete Working Examples

### Multimodal Inference with Mixed Backends

This example demonstrates running a multimodal conversation where vision processing happens on GPU while audio and text inference remain on CPU:

```python
import litert_lm

engine = litert_lm.Engine(
    model_path="my_model.litertlm",
    backend=litert_lm.Backend.CPU,
    vision_backend=litert_lm.Backend.GPU,
    audio_backend=litert_lm.Backend.CPU
)

with engine.create_conversation() as conv:
    message = {
        "role": "user",
        "content": [
            {"type": "image", "path": "scene.jpg"},
            {"type": "audio", "path": "speech.wav"},
            {"type": "text", "text": "Describe the scene and transcribe the audio."}
        ],
    }
    response = conv.send_message(message)
    print(response["content"][0]["text"])

```

### Benchmarking on Specific Hardware

To benchmark model performance on GPU from the command line:

```bash
litert-lm benchmark ./my_model.litertlm --backend=gpu --num_iterations=100

```

### Running Audio-Only Examples

The repository includes a multimodal example that demonstrates audio backend configuration:

```bash
python -m litert_lm.examples.multimodal_main \
    --model_path=my_model.litertlm \
    --audio_path=sample.wav

```

According to the source in [`python/litert_lm/examples/multimodal_main.py`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/python/litert_lm/examples/multimodal_main.py), this example internally configures `audio_backend=Backend.CPU` when initializing the engine.

## Summary

- **Backend Enum**: Hardware targets are defined in [`python/litert_lm/interfaces.py`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/python/litert_lm/interfaces.py) as `UNSPECIFIED`, `CPU`, and `GPU`.
- **Inference Backend**: Configure via CLI (`--backend cpu|gpu`) handled by [`python/litert_lm_cli/model.py`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/python/litert_lm_cli/model.py), or via Python `Engine(backend=...)` argument.
- **Encoder Backends**: Set `vision_backend` and `audio_backend` arguments in the Python API only; these default to `UNSPECIFIED` if not provided.
- **Model Constraints**: Enforce hardware restrictions during model packaging in [`schema/py/litertlm_builder.py`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/schema/py/litertlm_builder.py) using the `backend_constraint` parameter.
- **Defaults**: All backends default to CPU (inference) or auto-selection (encoders) when not explicitly configured.

## Frequently Asked Questions

### Can I use different backends for inference and encoders in the same model?

Yes. As implemented in [`python/litert_lm/interfaces.py`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/python/litert_lm/interfaces.py), the `Engine` class accepts separate `backend`, `vision_backend`, and `audio_backend` arguments. This allows you to run the main transformer on CPU while offloading vision encoders to GPU, which is useful for balancing memory constraints and compute latency.

### What happens if I set `vision_backend` to `UNSPECIFIED`?

Setting `vision_backend` or `audio_backend` to `Backend.UNSPECIFIED` (or omitting the argument) allows the LiteRT-LM engine to automatically select the appropriate hardware based on availability and model requirements. This differs from explicitly setting `Backend.CPU` or `Backend.GPU`, which forces execution to that specific hardware.

### How do I prevent my model from running on CPU?

During model packaging in [`schema/py/litertlm_builder.py`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/schema/py/litertlm_builder.py), specify `backend_constraint="gpu"` when calling `add_tflite_model()`. This embeds a whitelist into the model file that causes the runtime to reject CPU initialization attempts, ensuring the model only executes on compatible GPU hardware.

### Why can't I set vision or audio backends via the CLI?

The CLI interface in [`python/litert_lm_cli/main.py`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/python/litert_lm_cli/main.py) only exposes the `--backend` flag for the main inference engine. Vision and audio encoders require the Python API because they need to be configured as part of the `Engine` dataclass instantiation, as shown in the `AbstractEngine` definition in [`python/litert_lm/interfaces.py`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/python/litert_lm/interfaces.py).