LoRA (LyCORIS) Integration in AUTOMATIC1111: How Low-Rank Adapters Work in the Pipeline

LoRA (Low-Rank Adaptation) and its extended family LyCORIS are integrated into the AUTOMATIC1111 Stable Diffusion WebUI as a builtin extension that injects lightweight weight deltas into the base UNet during the forward pass, enabling runtime customization without modifying the original model weights.

The AUTOMATIC1111/stable-diffusion-webui repository implements LyCORIS (LoRA with Kronecker Product and Other Variants) as a first-class citizen in its generation pipeline. This architecture allows users to stack multiple low-rank adapters—ranging from classic LoRA to specialized variants like LoKr and OFT—to fine-tune model behavior with minimal memory overhead.

What is LoRA and LyCORIS?

LoRA (Low-Rank Adaptation)

LoRA is a parameter-efficient fine-tuning technique that decomposes weight updates into two smaller matrices (up and down projections). Instead of training full model weights, LoRA learns a rank-restricted delta that is added to the original frozen weights during inference. This approach drastically reduces storage requirements while maintaining fine-tuning flexibility.

LyCORIS Variants

LyCORIS expands the original LoRA concept to include alternative mathematical formulations for the weight delta:

• LoKr: Utilizes Kronecker products for the up-down decomposition
• OFT/BOFT: Applies orthogonal transforms (OFT) and block-wise variants (BOFT)
• GLora: Generalized LoRA with extended parameterization

All variants share the same runtime interface but differ in how calc_updown() constructs the weight delta from stored tensors.

How LyCORIS Integrates into the AUTOMATIC1111 Pipeline

The Extension Architecture

The LyCORIS system is implemented as a builtin extension located in extensions-builtin/Lora. Unlike external plugins, this code ships with the core repository and hooks directly into the model loading and processing subsystems. The extension registers multiple ModuleType classes—ModuleTypeLora, ModuleTypeLokr, ModuleTypeOFT, and ModuleTypeGLora—each capable of parsing distinct weight key patterns from .safetensors files.

Module Registration and Discovery

When the WebUI initializes, modules/processing.py coordinates with the network subsystem to scan models/LyCORIS/ for adapter files. Each discovered file triggers the creation of a NetworkModule subclass appropriate to its internal weight structure:

# Conceptual flow during model loading

from extensions_builtin.Lora import network_lora, network_lokr, network_oft

# ModuleType classes register themselves with the network loader

module_types = [
    network_lora.ModuleTypeLora(),
    network_lokr.ModuleTypeLokr(), 
    network_oft.ModuleTypeOFT(),
]

Runtime Injection Mechanism

During image generation, the NetworkModule.forward() method intercepts UNet activations and injects the computed delta. The canonical implementation in extensions-builtin/Lora/network_lora.py demonstrates this pattern:

# From NetworkModuleLora.forward (excerpt)

self.up_model.to(device=devices.device)
self.down_model.to(device=devices.device)
return y + self.up_model(self.down_model(x)) * self.multiplier() * self.calc_scale()

The multiplier() function applies the user-specified strength slider from the UI, while calc_scale() handles normalization. This injection occurs at every applicable layer during the diffusion forward pass.

Technical Implementation Details

Core LoRA Implementation

The file extensions-builtin/Lora/network_lora.py contains the NetworkModuleLora class, which handles classic LoRA weight keys (lora_up.weight, lora_down.weight). Its calc_updown() method reconstructs the full rank matrix by multiplying the up and down projections, then scales the result by the network multiplier.

Variant Implementations

Each LyCORIS variant resides in its own module with specialized tensor operations:

LoKr (Kronecker Product) in extensions-builtin/Lora/network_lokr.py:

from extensions_builtin.Lora.network_lokr import NetworkModuleLokr

# Weights contain lokr_w1_a, lokr_w1_b, lokr_w2_a, lokr_w2_b

module = NetworkModuleLokr(net, network.NetworkWeights(lokr_weights))
delta = module.calc_updown(orig_weight)  # Builds Kronecker product internally

OFT/BOFT in extensions-builtin/Lora/network_oft.py implements orthogonal transforms using ModuleTypeOFT, while GLora in extensions-builtin/Lora/network_glora.py provides generalized low-rank adaptation through NetworkModuleGLora.

Helper Utilities

The extensions-builtin/Lora/lyco_helpers.py file provides shared mathematical primitives including CP-decomposition, Kronecker product calculations, and factorization routines used across variants to rebuild weight deltas efficiently.

UI Integration and Processing Hooks

The extensions-builtin/Lora/ui_extra_networks_lora.py registers LyCORIS networks in the Extra Networks → LoRA tab, enabling checkbox activation and weight sliders. During processing, modules/processing.py (around line 782) passes optimization flags like opts.cache_fp16_weight to the LoRA subsystem when FP8 quantization is active, as defined in modules/shared_options.py (line 244).

Practical Usage Examples

Loading via the WebUI

1. Place a LyCORIS file (e.g., style_adapter.safetensors) in models/LyCORIS/
2. Refresh the Extra Networks → LoRA tab in the UI
3. Activate the checkbox and adjust the Weight slider (0.0 to 1.0+)

Manual Application in Scripts

import torch
from modules import devices, sd_models
from extensions_builtin.Lora.network_lora import NetworkModuleLora

# Assuming unet is loaded and weights dict contains LoRA tensors

weights = {"lora_up.weight": up_tensor, "lora_down.weight": down_tensor}
module = NetworkModuleLora(net, network.NetworkWeights(weights))

# Calculate and apply delta

orig_weight = unet.conv1.weight
delta = module.calc_updown(orig_weight)
new_weight = orig_weight + delta

Programmatic Activation

from modules import shared, processing, sd_models

# Reload base model weights

sd_models.reload_model_weights()

# Append to active extra networks (simplified illustration)

shared.opts.extra_networks.append(("lora", "style_adapter"))

Summary

• LyCORIS is not a separate model but a runtime decoration system that injects weight deltas into the base UNet during the forward pass.
• The AUTOMATIC1111 implementation supports multiple variants—LoRA, LoKr, OFT/BOFT, and GLora—through a unified NetworkModule interface in extensions-builtin/Lora/.
• Weight deltas are computed via calc_updown() and applied in forward() methods, scaled by user-controlled multipliers from the UI.
• The system resides in the builtin extension path, enabling automatic discovery of .safetensors files from models/LyCORIS/ and integration with the processing pipeline via modules/processing.py.

Frequently Asked Questions

What is the difference between LoRA and LyCORIS in AUTOMATIC1111?

LoRA is the original low-rank adaptation method using simple up-down matrix multiplication, while LyCORIS is an umbrella term encompassing LoRA plus extended variants like LoKr (Kronecker product), OFT (orthogonal transform), and GLora. In the AUTOMATIC1111 codebase, all are handled by the same builtin extension under extensions-builtin/Lora/, with each variant implemented as a specific NetworkModule subclass that determines how the weight delta is mathematically constructed.

Where should I place LyCORIS files in the AUTOMATIC1111 directory structure?

Place .safetensors files in the models/LyCORIS/ directory. The WebUI automatically scans this location during startup and populates the Extra Networks → LoRA tab. The extension code in ui_extra_networks_lora.py handles the UI registration, while the network loader in the core processing modules manages the runtime instantiation of the appropriate NetworkModule type based on the file's internal weight keys.

How does the WebUI apply multiple LoRA/LyCORIS adapters simultaneously?

The pipeline creates a NetworkModule instance for each activated adapter and sequentially applies their deltas during the UNet forward pass. Each module's forward() method computes y + delta * multiplier, where the delta is specific to that adapter's mathematical formulation (standard matrix product for LoRA, Kronecker product for LoKr, etc.). The cumulative effect is the sum of all active adapter modifications applied to the base model weights.

Does using LyCORIS variants require more VRAM than standard LoRA?

Memory usage remains efficient across all variants because only the small adapter weights are loaded into VRAM, not copies of the base model. The extensions-builtin/Lora/lyco_helpers.py utilities ensure that Kronecker and orthogonal transformations are computed on-demand during the forward pass. However, complex variants like LoKr may involve slightly more computation per layer compared to classic LoRA due to the Kronecker product calculation in NetworkModuleLokr.calc_updown().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}