# Using Cosmos-RL for Post-Training SANA-Image and SANA-Video: A Complete Guide

> Master post-training SANA-Image and SANA-Video with NVlabs Cosmos-RL. Learn efficient supervised fine-tuning and reinforcement learning using PEFT LoRA adapters and asynchronous reward scoring.

- Repository: [NVIDIA Research Projects/Sana](https://github.com/NVlabs/Sana)
- Tags: how-to-guide
- Published: 2026-05-19

---

**NVIDIA’s Cosmos-RL framework integrates directly with SANA’s modular diffusion stack through the [`train_scripts/sol_rl/train_sana.py`](https://github.com/NVlabs/Sana/blob/main/train_scripts/sol_rl/train_sana.py) entry point, enabling both supervised fine-tuning and reinforcement learning with PEFT LoRA adapters and asynchronous reward scoring.**

The NVlabs/Sana repository provides an efficient diffusion transformer for high-resolution image and video generation that can be optimized after initial training using reinforcement learning techniques. Using Cosmos-RL for post-training SANA models allows you to align outputs with human preferences through configurable reward functions like PickScore and CLIPScore. This integration leverages SANA’s native checkpoint format and modular architecture to support both 2D image and 3D video latents without requiring separate pipeline implementations.

## Architecture of the Cosmos-RL Integration

The integration reuses SANA’s diffusion model definitions from `diffusion/model/builder.MODELS` and post-training patches located in `diffusion/post_training/diffusers_patch`. This modular approach allows Cosmos-RL to inject RL algorithms into the existing sampling pipeline while maintaining compatibility with native SANA checkpoints.

### Core Components

Cosmos-RL provides three essential systems that interface with SANA’s infrastructure. **Configuration and launch utilities** supply the `cosmos-rl` CLI that instantiates models, data loaders, and reward services. **Reward-service support** enables asynchronous scoring through PickScore, CLIPScore, ImageReward, and HPSv2 APIs. **RL algorithms** including Flow-GRPO, DDRL, and Diffusion-NFT operate directly on the diffusion latent space without modifying the base model weights.

### Model Compatibility

SANA’s architecture in [`diffusion/model/nets/sana_multi_scale.py`](https://github.com/NVlabs/Sana/blob/main/diffusion/model/nets/sana_multi_scale.py) handles both 2D image and 3D video latents, making the same post-training code reusable across modalities. The training entry point in [`train_scripts/sol_rl/train_sana.py`](https://github.com/NVlabs/Sana/blob/main/train_scripts/sol_rl/train_sana.py) loads native checkpoints via `_resolve_native_checkpoint_source` and constructs text embeddings using `encode_sana_prompt` from [`diffusion/post_training/diffusers_patch/text_encode.py`](https://github.com/NVlabs/Sana/blob/main/diffusion/post_training/diffusers_patch/text_encode.py).

## Setting Up the Environment

Begin by installing the Cosmos-RL framework and downloading a native SANA checkpoint. The checkpoint loader supports both Hugging Face repositories and local paths specified via `SANA_NATIVE_MODEL_PATH`.

```bash

# Install Cosmos-RL

pip install "git+https://github.com/nvidia-cosmos/cosmos-rl.git"

# Download a native SANA checkpoint

python -m tools.download find_model \
    --model_path SANA_LinearFFN.pth \
    --model_source hf://yitongl/SANA_LinearFFN/SANA_LinearFFN.pth

```

## Post-Training Workflows

Cosmos-RL for post-training supports both supervised fine-tuning and reinforcement learning through unified configuration files under `configs/sol_rl/`. The default configuration in [`configs/sol_rl/sana.py`](https://github.com/NVlabs/Sana/blob/main/configs/sol_rl/sana.py) defines resolution settings (1024px), LoRA target modules, and rollout parameters.

### Supervised Fine-Tuning with LoRA

For parameter-efficient supervised fine-tuning, Cosmos-RL wraps the SANA transformer in a **PEFT LoRA adapter**. The configuration sets `preview_model` and `fullrollout_model` to PEFT mode while maintaining the base checkpoint frozen, enabling efficient updates with minimal memory overhead.

```bash
cosmos-rl \
  --config ./configs/sol_rl/sana-image-sft-lora.toml \
  cosmos_rl.tools.dataset.diffusers_dataset

```

### Reinforcement Learning with Diffusion-NFT

The Diffusion-NFT algorithm executes rollouts using `pipeline_with_logprob_sana` from [`diffusion/post_training/diffusers_patch/pipeline_with_logprob.py`](https://github.com/NVlabs/Sana/blob/main/diffusion/post_training/diffusers_patch/pipeline_with_logprob.py) to generate samples and compute log-probabilities for selection. The preset function `sana_diffusionnft_pickscore` creates ready-to-run configs that set `fullrollout_model` to PEFT mode and `sample.full_rollout_num` to best-of-n sampling.

```bash
cosmos-rl \
  --config ./configs/sol_rl/sana-image-nft.toml \
  cosmos_rl.tools.dataset.diffusion_nft

```

### Video Post-Training with OCR Rewards

Video post-training reuses the same rollout logic but processes 3D latents through the sampler. The dataset loader returns video frames (e.g., `.mp4` files with OCR annotations), and the pipeline handles temporal dimensions without code changes to the underlying transformer.

```bash
cosmos-rl \
  --config ./configs/sol_rl/sana-video-nft.toml \
  cosmos_rl.tools.dataset.diffusion_nft

```

## Optimizing Rollout Parameters

The training script includes automatic batch size selection to optimize GPU memory usage during rollouts. Use the `_auto_rollout_bsz` helper to calculate compatible batch sizes that evenly divide your total images per prompt, ensuring efficient parallel generation.

```python
from train_scripts.sol_rl.train_sana import _auto_rollout_bsz

# Calculate batch size for 48 images per prompt

rollout_bsz = _auto_rollout_bsz(num_image_per_prompt=48)  # Returns 12

```

## Monitoring and Logging

Training metrics and generated samples are logged through **WandB** and the `log_rollout_images` helper. On-disk logs are stored under `logs/` with subdirectories organized by experiment configuration, while the asynchronous reward services report scores independently of the main training loop.

## Summary

- **Cosmos-RL for post-training SANA** provides a unified framework for both supervised fine-tuning and reinforcement learning on image and video models through [`train_scripts/sol_rl/train_sana.py`](https://github.com/NVlabs/Sana/blob/main/train_scripts/sol_rl/train_sana.py).
- The integration supports **PEFT LoRA adapters** and compilation modes including **NVFP4** for accelerated inference, configured via [`configs/sol_rl/sana.py`](https://github.com/NVlabs/Sana/blob/main/configs/sol_rl/sana.py).
- **Reward services** operate asynchronously, supporting PickScore, CLIPScore, ImageReward, and HPSv2 without blocking the training loop.
- The same codebase handles 2D images and 3D videos through [`diffusion/model/nets/sana_multi_scale.py`](https://github.com/NVlabs/Sana/blob/main/diffusion/model/nets/sana_multi_scale.py) and the `pipeline_with_logprob_sana` sampler.
- Native checkpoint loading via `_resolve_native_checkpoint_source` ensures compatibility with original SANA model weights.

## Frequently Asked Questions

### What reward functions does Cosmos-RL support for SANA post-training?

Cosmos-RL supports asynchronous scoring through **PickScore**, **CLIPScore**, **ImageReward**, and **HPSv2**. These reward services run in separate processes, querying generated samples via the rollout pipeline and returning gradients or rankings to the RL algorithms without interrupting the main training loop.

### Can I use Cosmos-RL with both SANA image and video models?

Yes. The underlying architecture in [`diffusion/model/nets/sana_multi_scale.py`](https://github.com/NVlabs/Sana/blob/main/diffusion/model/nets/sana_multi_scale.py) handles both 2D image and 3D video latents. You use the same [`train_scripts/sol_rl/train_sana.py`](https://github.com/NVlabs/Sana/blob/main/train_scripts/sol_rl/train_sana.py) script for both modalities, changing only the dataset configuration to point to image directories or video files (`.mp4`), while the sampler automatically adapts to the latent dimensions.

### How does the PEFT LoRA integration work in the training pipeline?

The pipeline wraps the SANA transformer in a **PEFT LoRA adapter** when `preview_model` or `fullrollout_model` is set to `"peft"` in the configuration. This occurs after loading the native checkpoint via `_resolve_native_checkpoint_source`, allowing efficient fine-tuning with minimal memory overhead while keeping the base model frozen during rollouts.

### Where are the rollout logs and metrics stored during training?

Training metrics and generated samples are logged to **WandB** and saved on disk under the `logs/` directory. The `log_rollout_images` helper in the training script saves visualizations of rollouts for inspection, while experiment-specific configurations determine the exact subdirectory structure based on the algorithm and reward function used.