# Stable Diffusion Fine-Tuning: Complete Guide to Training Configuration Options in main.py

> Explore Stable Diffusion fine-tuning options in main.py. Master training configuration for learning rate scaling, GPU allocation, and experiment logging.

- Repository: [CompVis - Computer Vision and Learning LMU Munich/stable-diffusion](https://github.com/CompVis/stable-diffusion)
- Tags: deep-dive
- Published: 2026-03-02

---

**Stable Diffusion's [`main.py`](https://github.com/CompVis/stable-diffusion/blob/main/main.py) exposes all fine-tuning hyperparameters through a hybrid command-line and YAML configuration system, combining custom arguments in `get_parser()` with PyTorch Lightning's full trainer interface to control learning rate scaling, checkpoint resumption, GPU allocation, and experiment logging.**

The [`main.py`](https://github.com/CompVis/stable-diffusion/blob/main/main.py) script serves as the central orchestration point for fine-tuning Stable Diffusion models in the [CompVis/stable-diffusion](https://github.com/CompVis/stable-diffusion) repository. It stitches together the model architecture, data module, logger, callbacks, and the PyTorch Lightning trainer through a sophisticated configuration pipeline that merges command-line arguments with YAML base configs.

## Command-Line Arguments in get_parser()

The `get_parser()` function in [`main.py`](https://github.com/CompVis/stable-diffusion/blob/main/main.py) defines the custom argument parser that handles experiment-specific controls before Lightning trainer arguments are injected.

### Run Control and Experiment Naming

The script provides granular control over experiment identification and directory structure:

- **`-n / --name`** (`str`, default: `""`): Appends a post-fix to the automatic log directory name at [`main.py:L35-L44`](https://github.com/CompVis/stable-diffusion/blob/main/main.py#L35-L44), enabling distinction between multiple experimental runs.
- **`-p / --project`** (`str`): Specifies the project directory name passed directly to the logger configuration at [`main.py:L80-L84`](https://github.com/CompVis/stable-diffusion/blob/main/main.py#L80-L84).
- **`-l / --logdir`** (`str`, default: `"logs"`): Sets the base directory where all experiment folders are created, as implemented at [`main.py:L108-L114`](https://github.com/CompVis/stable-diffusion/blob/main/main.py#L108-L114).
- **`-f / --postfix`** (`str`, default: `""`): Adds an additional postfix to the automatically generated log-dir name beyond the `--name` parameter at [`main.py:L100-L106`](https://github.com/CompVis/stable-diffusion/blob/main/main.py#L100-L106).

### Resume and Checkpoint Management

Fine-tuning workflows depend heavily on checkpoint resumption capabilities:

- **`-r / --resume`** (`str`, default: `""`): Accepts a path to a previous log folder or specific checkpoint file to resume training from, parsed at [`main.py:L46-L53`](https://github.com/CompVis/stable-diffusion/blob/main/main.py#L46-L53). The script automatically detects whether the path points to a directory or a `.ckpt` file.
- **`-t / --train`** (`bool`, default: `False`): Acts as the training gatekeeper; only when set to `true` does the script invoke `trainer.fit()` at [`main.py:L64-L71`](https://github.com/CompVis/stable-diffusion/blob/main/main.py#L64-L71).
- **`--no-test`** (`bool`, default: `False`): When enabled, skips the final test run after training completion, defined at [`main.py:L73-L78`](https://github.com/CompVis/stable-diffusion/blob/main/main.py#L73-L78).

### Debugging and Reproducibility

For development and deterministic experimentation:

- **`-d / --debug`** (`bool`, default: `False`): Enables post-mortem debugging using `pudb` on uncaught exceptions at [`main.py:L86-L92`](https://github.com/CompVis/stable-diffusion/blob/main/main.py#L86-L92).
- **`-s / --seed`** (`int`, default: `23`): Sets the global random seed via `pl.seed_everything()` at [`main.py:L94-L99`](https://github.com/CompVis/stable-diffusion/blob/main/main.py#L94-L99), ensuring reproducible fine-tuning runs.

### Learning Rate Scaling

- **`--scale_lr`** (`bool`, default: `True`): Controls automatic learning rate scaling based on hardware configuration at [`main.py:L115-L122`](https://github.com/CompVis/stable-diffusion/blob/main/main.py#L115-L122). When enabled, the base learning rate is multiplied by `accumulate_grad_batches × num_gpus × batch_size`.

## PyTorch Lightning Trainer Integration

After the custom parser is constructed, `pytorch_lightning.Trainer.add_argparse_args(parser)` is called at [`main.py:L66-L68`](https://github.com/CompVis/stable-diffusion/blob/main/main.py#L66-L68) to inject the complete suite of Lightning trainer flags. This exposes critical distributed training and optimization parameters including:

- **`--gpus`**: GPU device indices or count for training
- **`--max_epochs`**: Maximum number of training epochs
- **`--accelerator`**: Backend selection (e.g., "gpu", "cpu", "tpu")
- **`--precision`**: Numerical precision (16, 32, or 64-bit)
- **`--accumulate_grad_batches`**: Gradient accumulation steps
- **`--gradient_clip_val`**: Gradient clipping threshold

These values are subsequently merged into `trainer_config` at lines ~1190-1225 and filtered through `nondefault_trainer_args()` at lines 1222-1230 to distinguish user-specified overrides from defaults.

## Configuration Merging and YAML Base Files

The configuration system employs OmegaConf to merge hierarchical YAML files with command-line overrides:

1. **Base Configuration Loading**: The `-b / --base` argument (type `list[str]`, default: `[]`) accepts one or more YAML config files at [`main.py:L55-L62`](https://github.com/CompVis/stable-diffusion/blob/main/main.py#L55-L62) that are merged left-to-right using `OmegaConf.load()`.
2. **Dotlist Overrides**: Additional `--key value` pairs are parsed via `OmegaConf.from_dotlist()` and merged on top of the base configs at lines 1145-1154.
3. **Final Config Object**: The merged configuration produces a unified object containing `model` and `data` subtrees that instantiate the LightningModule and DataModule respectively.

This architecture separates hyperparameters (learning rates, model architecture) from runtime parameters (GPUs, logging directories).

## Learning Rate Scaling Implementation

When `--scale_lr` is enabled, the script computes the effective learning rate at lines 862-889:

```python
model.learning_rate = accumulate_grad_batches * ngpu * batch_size * base_lr

```

This calculation directly modifies the `learning_rate` attribute of the model's LightningModule (typically defined in [`ldm/models/diffusion/ddpm.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/models/diffusion/ddpm.py)) before optimizer initialization, ensuring linear scaling rules are applied consistently across different hardware configurations.

## Callbacks and Logging Infrastructure

The parsed configuration options drive the instantiation of several critical callbacks at lines 1245-1270:

- **SetupCallback**: Handles directory creation for `logdir`, `ckptdir`, and `cfgdir` based on parsed paths.
- **ImageLogger**: Configures logging intervals and image visualization parameters.
- **LearningRateMonitor**: Tracks learning rate changes throughout training.
- **CUDACallback**: Manages GPU-specific optimizations and emptying caches.

The Trainer receives these callbacks alongside the logger configuration at lines 1257-1260, creating a fully instrumented training environment.

## Data Module Instantiation

The data configuration subtree is instantiated via `instantiate_from_config(config.data)` at line 1262, which dynamically imports and constructs the DataModule specified in the YAML configuration. Batch size, number of workers, and dataset-specific parameters flow from the merged config into the `ldm/data` classes.

## Practical Fine-Tuning Examples

The following command-line patterns demonstrate typical fine-tuning configurations using the training options available in [`main.py`](https://github.com/CompVis/stable-diffusion/blob/main/main.py):

**Basic fine-tuning with custom config and disabled LR scaling:**

```bash
python main.py \
  -b configs/stable-diffusion/v1-finetune.yaml \
  --scale_lr false \
  -t true \
  --seed 42 \
  --logdir ./my_runs \
  --name "my_finetune"

```

**Resume from checkpoint with multi-GPU training:**

```bash
python main.py \
  -b configs/base.yaml configs/extra.yaml \
  -r ./logs/2023-08-01T12-00-00_my_finetune/checkpoints/last.ckpt \
  -t true \
  --gpus 0,1,2,3 \
  --max_epochs 30

```

**Evaluation-only run (no training):**

```bash
python main.py \
  -b configs/eval.yaml \
  -r ./logs/2023-08-01T12-00-00_my_finetune/checkpoints/epoch=09.ckpt \
  --no-test false \
  -t false

```

## Summary

- **[`main.py`](https://github.com/CompVis/stable-diffusion/blob/main/main.py)** serves as the unified entry point for Stable Diffusion fine-tuning, exposing controls through `get_parser()` and PyTorch Lightning's argument interface.
- **Configuration merging** combines multiple YAML base files (`-b`) with command-line dotlist overrides using OmegaConf at lines 1145-1154.
- **Learning rate scaling** automatically adjusts base rates by `accumulate_grad_batches × GPUs × batch_size` when `--scale_lr` is enabled (lines 862-889).
- **Checkpoint resumption** supports both directory paths and specific `.ckpt` files via the `-r` argument at lines 46-53.
- **Trainer customization** leverages Lightning's full flag suite for distributed training, precision, and gradient management.

## Frequently Asked Questions

### How do I resume training from a specific checkpoint in Stable Diffusion?

Use the `-r / --resume` argument followed by either a path to a checkpoint file (`.ckpt`) or the log directory containing a `checkpoints` folder. According to the source code at [`main.py:L46-L53`](https://github.com/CompVis/stable-diffusion/blob/main/main.py#L46-L53), the script automatically detects the format and restores both model weights and optimizer states. You must also set `-t true` to continue training rather than just loading the model for inference.

### What is the difference between `--name` and `--postfix` in main.py?

The `--name` argument (lines 35-44) sets the primary experiment identifier used in the log directory structure, while `--postfix` (lines 100-106) appends an additional string after the automatic timestamp and name components. Use `--name` for meaningful experiment labels (e.g., "lr_1e-4_b32") and `--postfix` for minor variations or version notes that should appear in the directory name but remain distinct from the core experiment identity.

### How does the learning rate scaling work when fine-tuning?

When `--scale_lr` is set to `true` (the default at lines 115-122), the script calculates the effective learning rate as `base_lr × accumulate_grad_batches × num_gpus × batch_size` at lines 862-889. This linear scaling rule ensures that the optimization dynamics remain consistent regardless of whether you train on a single GPU with batch size 4 or eight GPUs with batch size 4 and gradient accumulation of 2.

### Can I override specific values in the YAML config from the command line?

Yes. After specifying base configs with `-b`, provide additional arguments as `--key value` pairs (e.g., `--model.params.lr_scheduler.target 5e-5`). The parser at lines 1145-1154 converts these to dotlist notation and merges them into the OmegaConf configuration object, overriding any values defined in the YAML files. This allows quick hyperparameter sweeps without creating multiple config files.