How the AUTOMATIC1111 Training Tab Works for Embeddings and Hypernetworks: A Technical Deep Dive

The AUTOMATIC1111 training tab routes user inputs through Gradio callbacks to either train_embedding() in modules/textual_inversion/textual_inversion.py or train_hypernetwork() in modules/hypernetworks/hypernetwork.py, both executing gradient-accumulating optimization loops with shared dataset loading, learning rate scheduling, and interruption handling.

The AUTOMATIC1111 Stable Diffusion WebUI training tab provides a unified interface for fine-tuning textual inversion embeddings and hypernetworks without writing custom training scripts. According to the source code in modules/ui.py, the tab constructs Gradio components that wire directly to specialized training pipelines handling dataset preparation, optimizer configuration, and checkpoint serialization. Whether you are teaching the model a new token representation via embeddings or injecting style-specific modulation layers via hypernetworks, the underlying architecture follows the same validation, training, and logging infrastructure defined in the repository’s core modules.

UI Architecture and Entry Points

The Train tab interface is constructed in modules/ui.py (approximately lines 968-990) using Gradio sliders, text boxes, and buttons. Key UI elements include dimension controls, learning rate inputs, and action buttons:

# modules/ui.py – UI construction for training controls

training_width = gr.Slider(minimum=64, maximum=2048, step=8,
                          label="Width", value=512, elem_id="train_training_width")
training_height = gr.Slider(minimum=64, maximum=2048, step=8,
                           label="Height", value=512, elem_id="train_training_height")
train_embedding = gr.Button(value="Train Embedding", variant='primary',
                           elem_id="train_train_embedding")
train_hypernetwork = gr.Button(value="Train Hypernetwork", variant='primary',
                               elem_id="train_train_hypernetwork")

Callback wiring occurs immediately after UI definition. The Train Embedding button triggers textual_inversion_ui.train_embedding, while Train Hypernetwork triggers hypernetworks_ui.train_hypernetwork. Both utilize wrap_gradio_gpu_call to ensure GPU computation runs on a separate thread while maintaining UI responsiveness:

# modules/ui.py – Embedding callback registration (lines 1037-1054)

train_embedding.click(
    fn=wrap_gradio_gpu_call(textual_inversion_ui.train_embedding,
                            extra_outputs=[gr.update()]),
    _js="start_training_textual_inversion",
    inputs=[ ... training parameters ... ],
    outputs=[ti_output, ti_outcome],
)

# modules/ui.py – Hypernetwork callback registration (lines 1071-1088)

train_hypernetwork.click(
    fn=wrap_gradio_gpu_call(hypernetworks_ui.train_hypernetwork,
                            extra_outputs=[gr.update()]),
    _js="start_training_textual_inversion",
    inputs=[ ... training parameters ... ],
    outputs=[ti_output, ti_outcome],
)

The Interrupt button connects directly to shared.state.interrupt(), allowing immediate termination of either training job by setting a global interruption flag checked within the training loops.

Training Textual Inversion Embeddings

Entry Point and Validation

When a user initiates embedding training, the request flows through textual_inversion_ui.train_embedding in modules/textual_inversion/ui.py. This wrapper performs two critical checks: it asserts that shared.cmd_opts.lowvram is disabled (training requires sufficient VRAM), and conditionally disables cross-attention optimizations if the user has opted to train without them:

# modules/textual_inversion/ui.py – Entry point (lines 17-28)

def train_embedding(*args):
    assert not shared.cmd_opts.lowvram, 'Training models with lowvram not possible'
    apply_optimizations = shared.opts.training_xattention_optimizations
    if not apply_optimizations:
        sd_hijack.undo_optimizations()
    embedding, filename = modules.textual_inversion.textual_inversion.train_embedding(*args)

Core Training Loop

The actual optimization occurs in modules/textual_inversion/textual_inversion.py within the train_embedding() function (signature at lines 400-410). The implementation follows a structured pipeline:

1. Input validation via validate_train_inputs()
2. Dataset preparation using PersonalizedBase, which handles image loading, optional PNG alpha-channel weighting, and tag shuffling
3. Optimizer and scheduler setup with LearnRateScheduler for dynamic learning rate adjustment
4. Gradient accumulation loop respecting the gradient_step parameter
5. Checkpointing and preview generation at user-specified intervals

The training loop utilizes automatic mixed precision and gradient scaling:

# modules/textual_inversion/textual_inversion.py – Training loop excerpt

scheduler = LearnRateScheduler(learn_rate, steps, initial_step)
clip_grad = torch.nn.utils.clip_grad_value_ if clip_grad_mode == "value" else \
            torch.nn.utils.clip_grad_norm_ if clip_grad_mode == "norm" else None

for _ in range((steps - initial_step) * gradient_step):
    if scheduler.finished or shared.state.interrupted:
        break
    for j, batch in enumerate(dl):
        with devices.autocast():
            x = batch.latent_sample.to(devices.device, non_blocking=pin_memory)
            loss = shared.sd_model.forward(x, c)[0] / gradient_step
        scaler.scale(loss).backward()
        if (j + 1) % gradient_step == 0:
            scaler.step(optimizer); scaler.update()
            if clip_grad:
                clip_grad(weights, clip_grad_sched.learn_rate)
            optimizer.zero_grad()

The function saves embeddings to the embedding_dir and generates preview images based on the create_image_every and save_embedding_every parameters. Upon completion, it returns the updated embedding object and file path to the UI.

Training Hypernetworks

Architectural Differences

Hypernetwork training in AUTOMATIC1111 follows a parallel architecture but implements fundamentally different model modifications. While embeddings learn new token vectors in the text encoder's embedding space, hypernetworks learn additional neural network layers that modulate the UNet's cross-attention mechanisms.

The entry point in modules/hypernetworks/ui.py mirrors the embedding wrapper but clears any previously loaded hypernetworks to ensure a clean training state:

# modules/hypernetworks/ui.py – Entry point (lines 17-28)

def train_hypernetwork(*args):
    shared.loaded_hypernetworks = []
    hypernetwork, filename = modules.hypernetworks.hypernetwork.train_hypernetwork(*args)
    return f"Training {'interrupted' if shared.state.interrupted else 'finished'} at {hypernetwork.step} steps.", ""

Hypernetwork Injection Mechanism

The critical distinction appears in modules/hypernetworks/hypernetwork.py (lines 473-485). The training function loads or creates a Hypernetwork instance and inserts it into shared.loaded_hypernetworks. During forward passes, the UNet's cross-attention layers are dynamically patched via sd_hijack_optimizations.py (lines 186-190) to invoke hypernetwork.apply_hypernetworks(), injecting learned modulation tensors into attention keys and values.

The dataset configuration differs by setting include_cond=True, providing the hypernetwork access to both image latents and text conditioning:

# modules/hypernetworks/hypernetwork.py – Dataset with conditioning

ds = modules.textual_inversion.dataset.PersonalizedBase(
    data_root=data_root, width=training_width, height=training_height,
    repeats=shared.opts.training_image_repeats_per_epoch,
    placeholder_token=hypernetwork_name,
    model=shared.sd_model, cond_model=shared.sd_model.cond_stage_model,
    device=devices.device, template_file=template_file,
    include_cond=True,  # Critical difference from embeddings

    batch_size=batch_size,
    gradient_step=gradient_step, shuffle_tags=shuffle_tags,
    tag_drop_out=tag_drop_out, latent_sampling_method=latent_sampling_method,
    varsize=varsize, use_weight=use_weight)

The optimization loop structure—gradient accumulation, clipping, and mixed precision—remains identical to the embedding pipeline, ensuring consistent behavior across training modes.

Shared Training Infrastructure

Both training modes rely on configuration options defined in modules/shared_options.py under the training category (lines 155-167). These settings control hardware utilization and logging behavior:

• unload_models_when_training: Moves VAE and CLIP to system RAM during training to maximize VRAM for the UNet
• pin_memory: Enables pin_memory=True for the DataLoader, accelerating CPU-to-GPU transfers
• training_xattention_optimizations: Determines whether to disable memory-efficient attention optimizations during training (required for numerical stability in some configurations)
• training_enable_tensorboard: Activates TensorBoard logging with configurable flush intervals and image saving

The training loops check shared.state.interrupted at every step boundary, allowing immediate termination via the UI's Interrupt button regardless of training mode.

Programmatic Training Without the UI

You can trigger training programmatically by importing the core functions directly, bypassing the Gradio interface. This approach is useful for automation or integration with external workflows:

import modules.textual_inversion.textual_inversion as ti
from modules import shared

# Configure training parameters

args = (
    "CustomConcept",        # embedding_name

    0.0005,                 # learn_rate

    4,                      # batch_size

    1,                      # gradient_step

    "/path/to/dataset",     # data_root

    "logs/embeddings",      # log_directory

    512, 512,               # training_width, training_height

    False,                  # varsize

    20000,                  # steps

    "norm", 0.1,            # clip_grad_mode, clip_grad_value

    True, False,            # shuffle_tags, tag_drop_out

    "once", False,          # latent_sampling_method, use_weight

    500, 500,               # create_image_every, save_embedding_every

    "style_filewords.txt",  # template_filename

    True,                   # save_image_with_stored_embedding

    False, "", "", 20, "Euler a", 7.0, 42, 512, 512   # preview params

)

# Execute training

embedding, filename = ti.train_embedding(*args)
print(f"Saved to {filename} at step {embedding.step}")

For hypernetworks, substitute modules.hypernetworks.hypernetwork.train_hypernetwork() with equivalent parameters.

Summary

• The Train tab in modules/ui.py constructs Gradio interfaces that route to specialized training functions via wrap_gradio_gpu_call().
• Embeddings train via modules/textual_inversion/textual_inversion.py, learning new token vectors using the PersonalizedBase dataset without UNet modification.
• Hypernetworks train via modules/hypernetworks/hypernetwork.py, learning attention modulation layers that are injected into the UNet through sd_hijack_optimizations.py.
• Both modes share gradient accumulation, learning rate scheduling, gradient clipping (value or norm), and interruption handling via shared.state.interrupted.
• Training configuration persists in modules/shared_options.py, controlling VRAM management, TensorBoard logging, and optimization settings.

Frequently Asked Questions

What is the difference between training embeddings and hypernetworks in AUTOMATIC1111?

Textual inversion embeddings learn new token vectors that extend the text encoder's vocabulary, allowing you to reference custom concepts in prompts using short trigger words. Hypernetworks learn additional neural network weights that modify the UNet's cross-attention layers, altering how the model renders styles or subjects without changing the base model weights. According to the source code, embeddings modify the conditioning phase while hypernetworks patch the diffusion forward pass via sd_hijack_optimizations.py.

How do I interrupt a training job in the AUTOMATIC1111 training tab?

Click the Interrupt button in the UI, which triggers shared.state.interrupt(). This sets a global boolean flag that the training loops in both textual_inversion.py and hypernetwork.py check at the start of every iteration. The loop breaks immediately, performs cleanup, and returns the current training state to the UI.

Where are training checkpoints saved in AUTOMATIC1111?

Embeddings save as .pt or .safetensors files to the directory specified by log_directory (defaulting to the embeddings folder), with filenames containing the step count. Hypernetworks save to the hypernetworks directory defined in the paths configuration. Both respect the save_embedding_every or save_hypernetwork_every parameters to control checkpoint frequency.

Can I train embeddings or hypernetworks without using the Gradio UI?

Yes. Import modules.textual_inversion.textual_inversion.train_embedding or modules.hypernetworks.hypernetwork.train_hypernetwork and call them with the appropriate argument tuple. Ensure shared.sd_model is loaded and shared.cmd_opts.lowvram is False, as the training code explicitly asserts sufficient VRAM availability before beginning optimization.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}