What Is Textual Inversion in AUTOMATIC1111 and How to Train Embeddings: A Complete Guide

Textual Inversion in AUTOMATIC1111 lets you teach Stable Diffusion new concepts by training a small embedding vector that represents a custom token, without modifying the base model weights.

Textual Inversion is a technique that extends the CLIP tokenizer's vocabulary with new, user-defined tokens. Instead of fine-tuning billions of diffusion model parameters, you optimize a tiny tensor—typically just a few vectors—that maps a placeholder string (like *mytoken*) to a visual concept. This allows you to generate personalized objects, styles, or faces using the original model's knowledge combined with your custom embedding stored separately on disk.

How Textual Inversion Works in AUTOMATIC1111

The implementation centers on a lightweight data pipeline that intercepts token processing without altering the U-Net or VAE weights. When you type a custom token into a prompt, the system replaces it with learned vectors before the text reaches the CLIP encoder.

The Embedding Data Structure

At the core is the Embedding class defined in [modules/textual_inversion/textual_inversion.py](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/textual_inversion/textual_inversion.py#L39-L86). This class holds:

• The learned tensor (the actual embedding vectors)
• The token name and checksum for validation
• Training step count and checkpoint metadata
• Methods for save() to disk and reloading

Each embedding file (.pt or .safetensors) stores this structure independently of the base model, making embeddings portable and lightweight.

Token Registration and the EmbeddingDatabase

The EmbeddingDatabase ([textual_inversion.py](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/textual_inversion/textual_inversion.py#L111-L158)) acts as a global registry that maps token IDs to embedding tensors. It watches the embeddings/ directory for file changes and hot-reloads new concepts without restarting the UI.

When processing prompts, the hijacked CLIP tokenizer in [modules/sd_hijack_clip.py](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/sd_hijack_clip.py#L12-L27) recognizes placeholder strings and injects the corresponding vectors at the correct offset in the token list. This allows the diffusion model to interpret your custom token as if it were part of the original vocabulary.

Model Integration

During startup, sd_models.load_textual_inversion_embeddings() automatically registers every file found in the embeddings folder. This makes previously trained concepts instantly available for inference without manual loading steps.

Creating Your First Embedding

Before training, you must initialize the embedding tensor. The create_embedding() function in [textual_inversion.py](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/textual_inversion/textual_inversion.py#L59-L84) allocates a zero-filled tensor (or copies a prototype from CLIP's existing vocabulary) and writes it to disk.

from modules.textual_inversion.textual_inversion import create_embedding

# Initialize a new token named "myperson"

# num_vectors_per_token=1 creates a single embedding vector

filename = create_embedding(
    name="myperson",
    num_vectors_per_token=1,
    overwrite_old=False,
    init_text='*'
)
print(f"Embedding initialized at: {filename}")

Key parameters:

• name: The token string you will type in prompts (e.g., *myperson*)
• num_vectors_per_token: Use 1 for simple concepts, or 2-16 for complex multi-vector embeddings that capture more detail
• init_text: Initializes the embedding using CLIP's encoding of this text (use * for random initialization)

Training Embeddings: The Full Pipeline

The train_embedding() function ([textual_inversion.py](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/textual_inversion/textual_inversion.py#L400-L889)) implements the complete optimization loop. It constructs a PersonalizedBase dataset from your image folder, runs the diffusion forward pass, and back-propagates gradients only through the embedding tensor—leaving the base model frozen.

from modules.textual_inversion.textual_inversion import train_embedding

train_embedding(
    id_task=None,
    embedding_name="myperson",
    learn_rate=5e-3,
    batch_size=1,
    gradient_step=1,
    data_root="training_data/myperson",  # Folder with your images

    log_directory="textual_inversion_logs",
    training_width=512,
    training_height=512,
    steps=3000,
    create_image_every=500,      # Generate preview every 500 steps

    save_embedding_every=500,    # Checkpoint every 500 steps

    template_filename="default.txt",
    shuffle_tags=True,
    tag_drop_out=0.1,
    preview_prompt="Portrait of *myperson*, high quality",
    preview_negative_prompt="blurry, lowres",
    preview_steps=20,
    preview_sampler_name="euler_a",
    preview_cfg_scale=7.0,
    preview_seed=-1,
    preview_width=512,
    preview_height=512,
)

Critical training mechanics:

• Dataset: The PersonalizedBase class in [dataset.py](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/textual_inversion/dataset.py) handles image cropping, caption parsing from filenames or text files, and template substitution
• Optimization: Only the embedding weights update; the diffusion U-Net remains static, keeping VRAM usage low and training fast
• Checkpointing: The function periodically calls save_embedding() to write intermediate versions to disk
• Logging: Training settings are written to a text file via [saving_settings.py](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/textual_inversion/saving_settings.py) for reproducibility

Training Parameters Explained

learn_rate: Typically 5e-3 to 1e-4. Higher values train faster but risk overfitting or vector collapse.

steps: 3000-10000 steps usually sufficient. The UI saves intermediate checkpoints based on save_embedding_every.

gradient_step: Accumulates gradients over N steps before updating. Effective for simulating larger batch sizes on limited VRAM.

shuffle_tags: Randomizes the order of comma-separated tags in captions, preventing the model from associating position with meaning.

Using Custom Tokens in Prompts

Once trained, trigger your concept using the token name wrapped in asterisks or as defined during creation:

prompt = "A fantasy portrait of *myperson* riding a dragon, 8k, highly detailed"

The hijacked tokenizer in [sd_hijack_clip.py](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/sd_hijack_clip.py#L12-L27) automatically substitutes *myperson* with your learned vectors before CLIP encoding occurs. You can combine multiple custom embeddings in one prompt and mix them with standard vocabulary.

File Structure and Configuration

AUTOMATIC1111 exposes several configuration points for the textual inversion pipeline:

Command-line options ([cmd_args.py](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/cmd_args.py#L34-L35)):

• --embeddings-dir: Relocate the storage folder from the default embeddings/
• --textual-inversion-templates-dir: Specify custom prompt templates for training

UI Settings ([shared_options.py](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/shared_options.py#L159-L160)):

• save_training_settings_to_txt: Writes hyperparameters alongside embeddings for version control
• textual_inversion_print_at_load: Logs embedding names to console on startup for debugging

Embeddings are stored as .pt or .safetensors files in the embeddings directory. Each file contains the tensor, metadata about the source checkpoint, and training step count, making them shareable and version-independent.

Summary

• Textual Inversion teaches new concepts by optimizing small embedding vectors rather than retraining the entire diffusion model, keeping training time to minutes instead of hours.
• The Embedding class and EmbeddingDatabase manage storage and registration, while [sd_hijack_clip.py](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/sd_hijack_clip.py) handles token substitution during inference.
• Use create_embedding() to initialize vectors and train_embedding() to run the optimization loop against your image dataset.
• Training modifies only the embedding weights via back-propagation, leaving the U-Net frozen and memory usage minimal.
• Configure storage locations via --embeddings-dir and enable reproducibility logging through the UI options in [shared_options.py](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/shared_options.py).

Frequently Asked Questions

How long does it take to train a Textual Inversion embedding in AUTOMATIC1111?

Training typically completes in 10-30 minutes on a single consumer GPU (RTX 3060/4060 or better). Because only the embedding tensor updates—usually containing 768 to 12,288 parameters rather than the model's billions—the optimization converges rapidly. The train_embedding() function runs for a user-specified step count (commonly 3000-5000 steps) and generates preview images periodically to monitor progress without manual intervention.

What is the difference between Textual Inversion and DreamBooth?

Textual Inversion learns a new token representation while keeping the base diffusion model completely frozen, resulting in small, portable files (a few kilobytes). DreamBooth fine-tunes the entire U-Net weights for a specific subject, producing larger checkpoint files (2-7GB) but often achieving better fidelity for complex subjects. Use Textual Inversion when you need lightweight, combinable concepts; use DreamBooth when you require maximum fidelity for a specific face or object and can afford the storage cost.

Can I use multiple embeddings in the same prompt?

Yes. The tokenizer processes each custom token independently and inserts their respective vectors into the prompt's embedding sequence. You can combine *person1* and *style2* in one prompt, and the model will attempt to render both concepts simultaneously. The EmbeddingDatabase supports unlimited concurrent embeddings, limited only by CLIP's maximum context length of 77 tokens.

Where are trained embeddings stored?

By default, embeddings save to the embeddings/ folder in your AUTOMATIC1111 installation. You can relocate this using the --embeddings-dir command-line argument defined in [cmd_args.py](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/cmd_args.py#L34-L35). Each embedding is a standalone .pt or .safetensors file containing the vector data and metadata, allowing you to share files between users or back up specific concepts without copying entire model checkpoints.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}