# LoRA Adapter Architecture in LiteRT-LM: Runtime Integration Deep Dive

> Explore the six-layer LoRA adapter architecture in LiteRT-LM. Learn how LiteRT-LM integrates LoRA for efficient runtime adaptation via memory-mapped files, regex validation, and manager components.

- Repository: [google-ai-edge/LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM)
- Tags: deep-dive
- Published: 2026-04-06

---

**TLDR:** LiteRT-LM implements Low-Rank Adaptation (LoRA) through a six-layer architecture that separates data ingestion via memory-mapped files, regex-based tensor name validation, adapter construction with TensorBuffer allocation, and session-scoped runtime management via ResourceManager and LoraManager components.

The google-ai-edge/LiteRT-LM repository implements a modular **LoRA adapter architecture** that enables efficient on-device fine-tuning for large language models. This architecture handles everything from raw weight file ingestion to session-specific adapter activation, allowing mobile and edge devices to switch between specialized model variants without reloading the base model. Understanding the interaction between `LoraData`, `LoraManager`, and `ResourceManager` is essential for deploying custom adapters in production environments.

## Data Ingestion Layer: LoraData

The `LoraData` class in [`runtime/util/lora_data.h`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/runtime/util/lora_data.h) serves as the primary interface for ingesting LoRA weight files, supporting both file-based and in-memory sources. It utilizes `MemoryMappedFileWithAutoAlignment` to create read-only views of tensor data without copying buffers into heap memory, crucial for mobile environments with limited RAM.

The class exposes three factory methods for different ingestion scenarios:

```cpp
// From a file path
static absl::StatusOr<std::unique_ptr<LoraData>> CreateFromFilePath(absl::string_view);

// From a ScopedFile (already opened)
static absl::StatusOr<std::unique_ptr<LoraData>> CreateFromScopedFile(std::shared_ptr<const ScopedFile>);

// From an existing BufferRef (in-memory)
static absl::StatusOr<std::unique_ptr<LoraData>> CreateFromBuffer(BufferRef<uint8_t>);

```

Once initialized, `LoraData` provides `ReadTensor()` for accessing individual weight tensors and `HasTensor()` for existence checks, both operating directly on the memory-mapped region in `runtime/util/lora_data.cc`.

## Tensor Name Validation

Before adapter construction, the system validates which model inputs correspond to LoRA tensors using the `IsLoRAInputName` function defined in `runtime/util/lora_util.cc`. LoRA tensors follow strict naming conventions such as `"lora_atten_q_a_prime_weight_0"` or `"query_w_prime_left_12"`.

The implementation compiles a regex pattern using `LazyRE2` to identify valid LoRA input names efficiently. This matcher runs during the adapter initialization phase to filter the model's input tensor list, ensuring only LoRA-specific buffers are overwritten while preserving base model inputs.

## Adapter Construction: The LoRA Component

The `LoRA` class in [`runtime/components/lora.h`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/runtime/components/lora.h) encapsulates the adapter state, mapping each validated LoRA tensor to a writable `litert::TensorBuffer`. Construction proceeds through `LoRA::Create()`, which accepts a `LoraData` instance and a reference to the compiled model.

The `Init()` method in `runtime/components/lora.cc` performs four critical steps for every model input:

1. **Validation**: Skips non-LoRA inputs using `IsLoRAInputName`.
2. **Allocation**: Creates writable input buffers via `CreateInputBuffer()`.
3. **Population**: Locks each buffer and either copies LoRA tensor data using `memcpy` or zero-fills absent tensors.
4. **Registration**: Stores filled buffers in the `lora_buffers_` map keyed by input name.

This approach ensures that missing LoRA weights default to zero tensors rather than uninitialized memory, maintaining model stability.

## Adapter Management via LoraManager

The **LoraManager** in [`runtime/components/lora_manager.h`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/runtime/components/lora_manager.h) maintains the lifecycle of multiple adapter instances through two internal maps:

- `lora_data_`: Temporarily stores raw `LoraData` objects
- `loras_`: Holds materialized `LoRA` objects ready for inference

**Loading** occurs via `LoadLoRA()`, which validates unique IDs, creates `ScopedFile` instances from `ModelAssets`, and populates `lora_data_`. **Activation** happens through `UseLoRA()`, which converts raw data into `LoRA` objects, moves them to active storage, and updates `current_lora_id_`. The `GetLoRABuffers()` method returns the currently active adapter's tensor buffers for binding to model inputs.

While typically invoked internally during session creation, the explicit activation flow demonstrates how adapters transition from loaded to active:

```cpp
// Inside the executor implementation (e.g., LockedLlmExecutor)
RETURN_IF_ERROR(llm_executor_->UseLoRA(lora_id.value()));

```

## Runtime Integration and Resource Management

The **ResourceManager** in `runtime/framework/resource_management/resource_manager.cc` bridges high-level executor APIs with the `LoraManager` implementation. It exposes `LoadLoRA()` to the executor and manages adapter ID assignment through `AssignLoraId()`.

The ID assignment logic supports two modes:
- **Path-based**: Hashing the LoRA file path to a `uint32_t` for reuse across sessions
- **Scoped file**: Generating temporary IDs for in-memory adapters without persistence

The mapping is maintained in `lora_hash_to_id_`, a `flat_hash_map<string, uint32_t>` that ensures globally unique identifiers.

### Loading from File Paths

The standard workflow for file-based adapters involves assigning an ID and loading assets:

```cpp
#include "runtime/framework/resource_management/resource_manager.h"
#include "runtime/util/lora_data.h"

// Assume `resource_manager` is a previously created ResourceManager instance

// 1. Assign a LoRA ID (path-based or scoped file)
std::optional<uint32_t> lora_id = resource_manager.AssignLoraId(
    /*lora_path=*/"/path/to/my_lora.tflite",
    /*has_scoped_lora_file=*/false);

// 2. Load the LoRA into the executor if needed
if (lora_id.has_value()) {
  // Build ModelAssets from the file (the helper creates a ScopedFile internally)
  ASSIGN_OR_RETURN(auto model_assets,
                   ModelAssets::Create(
                       ScopedFile::Open("/path/to/my_lora.tflite"),
                       /*model_path=*/""));
  // Forward to the executor (via ResourceManager)
  RETURN_IF_ERROR(resource_manager.LoadLoRA(lora_id.value(), model_assets));
}

```

### Loading In-Memory Adapters

For scenarios where LoRA weights reside in memory rather than disk, use the scoped file approach:

```cpp
// Create a ScopedFile from binary data already in memory
auto scoped_file = ScopedFile::CreateFromBuffer(my_lora_bytes);

// Build ModelAssets directly from the scoped file
auto model_assets = ModelAssets::Create(scoped_file, /*model_path=*/"");

// Assign a temporary ID (no path, only scoped file)
auto lora_id = resource_manager.AssignLoraId(/*lora_path=*/"", /*has_scoped_lora_file=*/true);
RETURN_IF_ERROR(resource_manager.LoadLoRA(lora_id.value(), model_assets));

```

## Session Management and Inference Execution

Each inference session maintains its LoRA context through the **ContextHandler** class in [`runtime/framework/resource_management/context_handler.h`](https://github.com/google-ai-edge/LiteRT-LM/blob/main/runtime/framework/resource_management/context_handler.h), which stores the active `lora_id` alongside runtime configuration. This design enables concurrent sessions with different adapters while sharing the base model weights.

During inference, the executor automatically retrieves LoRA buffers from `LoraManager::GetLoRABuffers()` and binds them to model inputs. Client code requires no additional steps beyond standard inference calls:

```cpp
// Prepare regular input tensors (e.g., token IDs) as usual
ExecutorInputs inputs = PrepareTextInputs(...);

// The executor automatically injects the LoRA buffers before the model
// runs because LoRA::Init() has already populated the input buffers
absl::StatusOr<std::vector<std::vector<int>>> output = executor->Decode();

```

## Summary

- **Memory Efficiency**: `LoraData` uses memory-mapped files (`MemoryMappedFileWithAutoAlignment`) to avoid copying weight data into RAM.
- **Validation Layer**: `IsLoRAInputName` employs compiled regex (`LazyRE2`) to identify LoRA tensors by naming convention.
- **Buffer Management**: The `LoRA` class creates writable `TensorBuffer` instances for each validated tensor, zero-filling missing weights.
- **Lifecycle Control**: `LoraManager` separates loading (`LoadLoRA`) from activation (`UseLoRA`), tracking state in `lora_data_` and `loras_` maps.
- **Session Isolation**: `ResourceManager` assigns unique IDs via `AssignLoraId` and stores them in `ContextHandler`, enabling multi-adapter concurrency via `lora_hash_to_id_`.
- **Zero-Copy Inference**: Active LoRA buffers attach automatically during executor calls, requiring no manual tensor manipulation.

## Frequently Asked Questions

### What file formats does LiteRT-LM support for LoRA weights?

LiteRT-LM supports LoRA weights in TFLite format and raw binary blobs. The `LoraData` class provides factory methods for file paths (`CreateFromFilePath`), pre-opened scoped files (`CreateFromScopedFile`), and in-memory buffers (`CreateFromBuffer`), all utilizing memory-mapped access for efficiency.

### How does the runtime handle missing LoRA tensors?

During `LoRA::Init()`, the system checks for tensor existence using `LoraData::HasTensor()`. If a specific LoRA weight is absent, the corresponding `TensorBuffer` is zero-filled rather than left uninitialized. This ensures the model receives valid input tensors while effectively applying zero adaptation to those parameters.

### Can multiple LoRA adapters be active simultaneously?

While only one LoRA adapter can be active per inference session, the architecture supports multiple concurrent sessions with different adapters. Each session's `ContextHandler` stores a unique LoRA ID, and `ResourceManager` caches loaded adapters in `lora_hash_to_id_`, allowing efficient switching without reloading weights.

### Where does the naming convention for LoRA tensors originate?

The naming patterns (e.g., `"lora_atten_q_a_prime_weight_0"`) are validated by the `IsLoRAInputName` function in `runtime/util/lora_util.cc`. The implementation uses a `LazyRE2` regex matcher to identify tensors that should receive LoRA weights versus base model inputs, ensuring only the adaptation parameters are overwritten during inference.