# LoRA (LyCORIS) Integration in AUTOMATIC1111: How Low-Rank Adapters Work in the Pipeline

> Learn how LoRA LyCORIS integrates into AUTOMATIC1111 for lightweight UNet customization. Adapt models at runtime without changing original weights. Master LoRA in Stable Diffusion.

- Repository: [AUTOMATIC1111/stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui)
- Tags: deep-dive
- Published: 2026-02-24

---

**LoRA (Low-Rank Adaptation) and its extended family LyCORIS are integrated into the AUTOMATIC1111 Stable Diffusion WebUI as a builtin extension that injects lightweight weight deltas into the base UNet during the forward pass, enabling runtime customization without modifying the original model weights.**

The AUTOMATIC1111/stable-diffusion-webui repository implements **LyCORIS** (LoRA with Kronecker Product and Other Variants) as a first-class citizen in its generation pipeline. This architecture allows users to stack multiple low-rank adapters—ranging from classic LoRA to specialized variants like LoKr and OFT—to fine-tune model behavior with minimal memory overhead.

## What is LoRA and LyCORIS?

### LoRA (Low-Rank Adaptation)

**LoRA** is a parameter-efficient fine-tuning technique that decomposes weight updates into two smaller matrices (up and down projections). Instead of training full model weights, LoRA learns a rank-restricted delta that is added to the original frozen weights during inference. This approach drastically reduces storage requirements while maintaining fine-tuning flexibility.

### LyCORIS Variants

**LyCORIS** expands the original LoRA concept to include alternative mathematical formulations for the weight delta:

- **LoKr**: Utilizes Kronecker products for the up-down decomposition
- **OFT/BOFT**: Applies orthogonal transforms (OFT) and block-wise variants (BOFT)
- **GLora**: Generalized LoRA with extended parameterization

All variants share the same runtime interface but differ in how `calc_updown()` constructs the weight delta from stored tensors.

## How LyCORIS Integrates into the AUTOMATIC1111 Pipeline

### The Extension Architecture

The LyCORIS system is implemented as a **builtin extension** located in `extensions-builtin/Lora`. Unlike external plugins, this code ships with the core repository and hooks directly into the model loading and processing subsystems. The extension registers multiple `ModuleType` classes—`ModuleTypeLora`, `ModuleTypeLokr`, `ModuleTypeOFT`, and `ModuleTypeGLora`—each capable of parsing distinct weight key patterns from `.safetensors` files.

### Module Registration and Discovery

When the WebUI initializes, [`modules/processing.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/processing.py) coordinates with the network subsystem to scan `models/LyCORIS/` for adapter files. Each discovered file triggers the creation of a `NetworkModule` subclass appropriate to its internal weight structure:

```python

# Conceptual flow during model loading

from extensions_builtin.Lora import network_lora, network_lokr, network_oft

# ModuleType classes register themselves with the network loader

module_types = [
    network_lora.ModuleTypeLora(),
    network_lokr.ModuleTypeLokr(), 
    network_oft.ModuleTypeOFT(),
]

```

### Runtime Injection Mechanism

During image generation, the `NetworkModule.forward()` method intercepts UNet activations and injects the computed delta. The canonical implementation in [`extensions-builtin/Lora/network_lora.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/extensions-builtin/Lora/network_lora.py) demonstrates this pattern:

```python

# From NetworkModuleLora.forward (excerpt)

self.up_model.to(device=devices.device)
self.down_model.to(device=devices.device)
return y + self.up_model(self.down_model(x)) * self.multiplier() * self.calc_scale()

```

The `multiplier()` function applies the user-specified strength slider from the UI, while `calc_scale()` handles normalization. This injection occurs at every applicable layer during the diffusion forward pass.

## Technical Implementation Details

### Core LoRA Implementation

The file [`extensions-builtin/Lora/network_lora.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/extensions-builtin/Lora/network_lora.py) contains the `NetworkModuleLora` class, which handles classic LoRA weight keys (`lora_up.weight`, `lora_down.weight`). Its `calc_updown()` method reconstructs the full rank matrix by multiplying the up and down projections, then scales the result by the network multiplier.

### Variant Implementations

Each LyCORIS variant resides in its own module with specialized tensor operations:

**LoKr (Kronecker Product)** in [`extensions-builtin/Lora/network_lokr.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/extensions-builtin/Lora/network_lokr.py):

```python
from extensions_builtin.Lora.network_lokr import NetworkModuleLokr

# Weights contain lokr_w1_a, lokr_w1_b, lokr_w2_a, lokr_w2_b

module = NetworkModuleLokr(net, network.NetworkWeights(lokr_weights))
delta = module.calc_updown(orig_weight)  # Builds Kronecker product internally

```

**OFT/BOFT** in [`extensions-builtin/Lora/network_oft.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/extensions-builtin/Lora/network_oft.py) implements orthogonal transforms using `ModuleTypeOFT`, while **GLora** in [`extensions-builtin/Lora/network_glora.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/extensions-builtin/Lora/network_glora.py) provides generalized low-rank adaptation through `NetworkModuleGLora`.

### Helper Utilities

The [`extensions-builtin/Lora/lyco_helpers.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/extensions-builtin/Lora/lyco_helpers.py) file provides shared mathematical primitives including CP-decomposition, Kronecker product calculations, and factorization routines used across variants to rebuild weight deltas efficiently.

### UI Integration and Processing Hooks

The [`extensions-builtin/Lora/ui_extra_networks_lora.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/extensions-builtin/Lora/ui_extra_networks_lora.py) registers LyCORIS networks in the **Extra Networks → LoRA** tab, enabling checkbox activation and weight sliders. During processing, [`modules/processing.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/processing.py) (around line 782) passes optimization flags like `opts.cache_fp16_weight` to the LoRA subsystem when FP8 quantization is active, as defined in [`modules/shared_options.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/shared_options.py) (line 244).

## Practical Usage Examples

### Loading via the WebUI

1. Place a LyCORIS file (e.g., `style_adapter.safetensors`) in `models/LyCORIS/`
2. Refresh the **Extra Networks → LoRA** tab in the UI
3. Activate the checkbox and adjust the **Weight** slider (0.0 to 1.0+)

### Manual Application in Scripts

```python
import torch
from modules import devices, sd_models
from extensions_builtin.Lora.network_lora import NetworkModuleLora

# Assuming unet is loaded and weights dict contains LoRA tensors

weights = {"lora_up.weight": up_tensor, "lora_down.weight": down_tensor}
module = NetworkModuleLora(net, network.NetworkWeights(weights))

# Calculate and apply delta

orig_weight = unet.conv1.weight
delta = module.calc_updown(orig_weight)
new_weight = orig_weight + delta

```

### Programmatic Activation

```python
from modules import shared, processing, sd_models

# Reload base model weights

sd_models.reload_model_weights()

# Append to active extra networks (simplified illustration)

shared.opts.extra_networks.append(("lora", "style_adapter"))

```

## Summary

- **LyCORIS is not a separate model** but a runtime decoration system that injects weight deltas into the base UNet during the forward pass.
- The AUTOMATIC1111 implementation supports multiple variants—**LoRA, LoKr, OFT/BOFT, and GLora**—through a unified `NetworkModule` interface in `extensions-builtin/Lora/`.
- Weight deltas are computed via `calc_updown()` and applied in `forward()` methods, scaled by user-controlled multipliers from the UI.
- The system resides in the builtin extension path, enabling automatic discovery of `.safetensors` files from `models/LyCORIS/` and integration with the processing pipeline via [`modules/processing.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/processing.py).

## Frequently Asked Questions

### What is the difference between LoRA and LyCORIS in AUTOMATIC1111?

**LoRA is the original low-rank adaptation method using simple up-down matrix multiplication, while LyCORIS is an umbrella term encompassing LoRA plus extended variants like LoKr (Kronecker product), OFT (orthogonal transform), and GLora.** In the AUTOMATIC1111 codebase, all are handled by the same builtin extension under `extensions-builtin/Lora/`, with each variant implemented as a specific `NetworkModule` subclass that determines how the weight delta is mathematically constructed.

### Where should I place LyCORIS files in the AUTOMATIC1111 directory structure?

**Place `.safetensors` files in the `models/LyCORIS/` directory.** The WebUI automatically scans this location during startup and populates the **Extra Networks → LoRA** tab. The extension code in [`ui_extra_networks_lora.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/ui_extra_networks_lora.py) handles the UI registration, while the network loader in the core processing modules manages the runtime instantiation of the appropriate `NetworkModule` type based on the file's internal weight keys.

### How does the WebUI apply multiple LoRA/LyCORIS adapters simultaneously?

**The pipeline creates a `NetworkModule` instance for each activated adapter and sequentially applies their deltas during the UNet forward pass.** Each module's `forward()` method computes `y + delta * multiplier`, where the delta is specific to that adapter's mathematical formulation (standard matrix product for LoRA, Kronecker product for LoKr, etc.). The cumulative effect is the sum of all active adapter modifications applied to the base model weights.

### Does using LyCORIS variants require more VRAM than standard LoRA?

**Memory usage remains efficient across all variants because only the small adapter weights are loaded into VRAM, not copies of the base model.** The [`extensions-builtin/Lora/lyco_helpers.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/extensions-builtin/Lora/lyco_helpers.py) utilities ensure that Kronecker and orthogonal transformations are computed on-demand during the forward pass. However, complex variants like LoKr may involve slightly more computation per layer compared to classic LoRA due to the Kronecker product calculation in `NetworkModuleLokr.calc_updown()`.