# How to Implement Few-Step DMD Distillation for Faster Inference in LongLive

> Learn how to implement few-step DMD distillation in LongLive for faster inference. Reduce num train timestep and use the score distillation trainer to compress the diffusion schedule.

- Repository: [NVIDIA Research Projects/LongLive](https://github.com/NVlabs/LongLive)
- Tags: how-to-guide
- Published: 2026-05-24

---

**To implement few-step DMD distillation in LongLive, reduce the `num_train_timestep` configuration parameter from 1000 to a smaller value (e.g., 100) while using the `score_distillation` trainer, which enables the `DMD` class to sample from a compressed diffusion schedule during training.**

LongLive is an open-source video generation framework by NVlabs that uses **Distribution Matching Distillation (DMD)** to train efficient student generators from teacher diffusion models. While standard DMD uses approximately 1000 diffusion timesteps for high-quality distillation, you can configure **few-step DMD distillation** to reduce inference time by 10× or more. This implementation requires modifying only the diffusion schedule parameters in [`model/dmd.py`](https://github.com/NVlabs/LongLive/blob/main/model/dmd.py) and [`utils/config.py`](https://github.com/NVlabs/LongLive/blob/main/utils/config.py), with no changes needed to the core loss computation.

## Understanding the DMD Architecture and Schedule Parameters

The few-step distillation capability centers on the `DMD` class in [`model/dmd.py`](https://github.com/NVlabs/LongLive/blob/main/model/dmd.py) (lines 13-31, 132-166), which wraps the student generator, real-score network, and fake-score network. During training, this class samples timesteps from a configurable diffusion trajectory and computes the KL-gradient loss between teacher and student predictions.

Key methods governing the diffusion process include:
- **`_compute_kl_grad`** (lines 92-134): Implements the teacher-student KL gradient matching from the original DMD paper.
- **`compute_distribution_matching_loss`** (lines 166-226): Converts the raw KL gradient into the final optimization objective.
- **`generator_loss` and `critic_loss`** (lines 236-298, 300-378): Handle forward passes, timestep sampling via `_get_timestep`, and optional critic training.

The diffusion schedule itself is defined by four critical parameters read during `DMD.__init__` (lines 42-45):
- **`num_train_timestep`**: Total number of diffusion steps in the trajectory (default 1000).
- **`min_step` and `max_step`**: Percentage bounds (typically 2 and 98) that constrain sampling to avoid boundary errors.
- **`ts_schedule` and `ts_schedule_max`**: Boolean flags that bias trajectory sampling toward specific timestep ranges.

Because the `DMD` class dynamically reads these values at initialization, reducing `num_train_timestep` automatically compresses the entire diffusion trajectory while preserving the mathematical correctness of the loss functions.

## Configuring a Few-Step Schedule in utils/config.py

When your configuration specifies `trainer: score_distillation`, the loader in [`utils/config.py`](https://github.com/NVlabs/LongLive/blob/main/utils/config.py) (lines 45-61) injects default DMD hyperparameters. To enable few-step distillation, override the `num_train_timestep` value in your YAML configuration file:

```yaml
model_name: "wan_t2v_A14B"
trainer: score_distillation
num_train_timestep: 100      # Compress from 1000 to 100 steps

ts_schedule: true
ts_schedule_max: false
teacher_forcing: true        # Recommended for few-step convergence

min_step: 2                  # 2% of 100 = step 2

max_step: 98                 # 98% of 100 = step 98

```

*Why this works*: The `DMD` class reads `num_train_timestep` during initialization and uses it to bound the random timestep sampler in `_get_timestep`. By shrinking this value from 1000 to 100, you restrict the student to learn only a **few-step diffusion trajectory**, eliminating the need for full-width sampling during inference.

### Optional Speed-Up Parameters

For optimal few-step convergence, consider adjusting these additional parameters defined in [`utils/config.py`](https://github.com/NVlabs/LongLive/blob/main/utils/config.py):

- **`teacher_forcing`**: Set to `true` to force the teacher network to generate latents matching the student’s trajectory, which stabilizes training when using fewer timesteps.
- **`backward_simulation`**: Keep as `true` (default) to maintain the backward-sampled generator input required for proper DMD gradient flow.
- **`real_guidance_scale` and `fake_guidance_scale`**: Maintain default values (`3.0` and `0.0`) unless experimenting with altered gradient magnitudes for specific few-step regimes.

## Launching Few-Step Distillation Training

Execute the distillation process using the standard training entry point:

```bash
python train.py --config configs/few_step.yaml

```

The [`train.py`](https://github.com/NVlabs/LongLive/blob/main/train.py) script detects `config.trainer == "score_distillation"` and constructs a `ScoreDistillationTrainer` (imported from [`trainer/distillation.py`](https://github.com/NVlabs/LongLive/blob/main/trainer/distillation.py)), which instantiates the `DMD` class with your compressed schedule. During training, verify that timestep values logged under the `timestep` metric remain within your configured bounds (e.g., 2-98), confirming that the few-step schedule is active.

## Running Fast Inference with Distilled Checkpoints

After training completes, deploy the distilled student using the standard inference scripts:

```bash
python inference.py \
  --checkpoint path/to/few_step_checkpoint.pt \
  --prompt "A futuristic cityscape at sunset" \
  --num_frames 16 \
  --output fast_generation.mp4

```

Because the student was trained specifically on a **few-step diffusion schedule**, the sampling loop executes in approximately **10× less time** than the full 1000-step configuration while retaining the visual quality of the original DMD distillation.

## Summary

- **Reduce `num_train_timestep`** in your configuration (e.g., from 1000 to 100) to enable few-step DMD distillation.
- The `DMD` class in [`model/dmd.py`](https://github.com/NVlabs/LongLive/blob/main/model/dmd.py) automatically adapts its `_get_timestep` sampler and loss computation to the compressed schedule without requiring code changes.
- Use `trainer: score_distillation` to trigger the `ScoreDistillationTrainer`, which initializes the DMD pipeline with your custom schedule parameters.
- Enable **`teacher_forcing: true`** for improved convergence when training with fewer than 200 timesteps.
- Inference uses the same scripts ([`inference.py`](https://github.com/NVlabs/LongLive/blob/main/inference.py) or [`inference_sp.py`](https://github.com/NVlabs/LongLive/blob/main/inference_sp.py)) but completes in significantly fewer steps, delivering faster generation.

## Frequently Asked Questions

### What is the optimal number of timesteps for few-step DMD in LongLive?

Most implementations use between 50 and 200 timesteps when configuring few-step DMD distillation, with **100 steps** serving as the optimal balance between speed and quality. This configuration typically reduces inference time by 90% compared to the default 1000-step schedule while maintaining visual fidelity comparable to the full teacher model.

### Does reducing timesteps affect the DMD loss computation in model/dmd.py?

No, the core DMD mathematics remains unchanged; the `_compute_kl_grad` method (lines 92-134) and `compute_distribution_matching_loss` method (lines 166-226) function identically regardless of schedule length. The adaptation occurs purely in the timestep sampling logic within `DMD.__init__` (lines 42-45) and `_get_timestep`, which respect the new `num_train_timestep` bounds automatically.

### Where are the default DMD hyperparameters defined in the LongLive repository?

Default DMD parameters are injected in [`utils/config.py`](https://github.com/NVlabs/LongLive/blob/main/utils/config.py) between lines 45 and 61 whenever the configuration specifies `trainer: score_distillation`. This block sets `num_train_timestep: 1000`, `min_step: 2`, `max_step: 98`, and guidance scale defaults, all of which you can override in your custom YAML configuration without modifying the source code.

### Can few-step DMD distillation work with video models other than Wan-T2V?

Yes, the few-step schedule configuration is architecture-agnostic and functions with any diffusion backbone supported by LongLive, including various Wan-T2V sizes and other video generation architectures. The `num_train_timestep` parameter affects only the trajectory length during distillation, not the underlying student generator architecture or the inference pipeline in [`inference.py`](https://github.com/NVlabs/LongLive/blob/main/inference.py).