Using Cosmos-RL for Post-Training SANA-Image and SANA-Video: A Complete Guide

NVIDIA’s Cosmos-RL framework integrates directly with SANA’s modular diffusion stack through the train_scripts/sol_rl/train_sana.py entry point, enabling both supervised fine-tuning and reinforcement learning with PEFT LoRA adapters and asynchronous reward scoring.

The NVlabs/Sana repository provides an efficient diffusion transformer for high-resolution image and video generation that can be optimized after initial training using reinforcement learning techniques. Using Cosmos-RL for post-training SANA models allows you to align outputs with human preferences through configurable reward functions like PickScore and CLIPScore. This integration leverages SANA’s native checkpoint format and modular architecture to support both 2D image and 3D video latents without requiring separate pipeline implementations.

Architecture of the Cosmos-RL Integration

The integration reuses SANA’s diffusion model definitions from diffusion/model/builder.MODELS and post-training patches located in diffusion/post_training/diffusers_patch. This modular approach allows Cosmos-RL to inject RL algorithms into the existing sampling pipeline while maintaining compatibility with native SANA checkpoints.

Core Components

Cosmos-RL provides three essential systems that interface with SANA’s infrastructure. Configuration and launch utilities supply the cosmos-rl CLI that instantiates models, data loaders, and reward services. Reward-service support enables asynchronous scoring through PickScore, CLIPScore, ImageReward, and HPSv2 APIs. RL algorithms including Flow-GRPO, DDRL, and Diffusion-NFT operate directly on the diffusion latent space without modifying the base model weights.

Model Compatibility

SANA’s architecture in diffusion/model/nets/sana_multi_scale.py handles both 2D image and 3D video latents, making the same post-training code reusable across modalities. The training entry point in train_scripts/sol_rl/train_sana.py loads native checkpoints via _resolve_native_checkpoint_source and constructs text embeddings using encode_sana_prompt from diffusion/post_training/diffusers_patch/text_encode.py.

Setting Up the Environment

Begin by installing the Cosmos-RL framework and downloading a native SANA checkpoint. The checkpoint loader supports both Hugging Face repositories and local paths specified via SANA_NATIVE_MODEL_PATH.

# Install Cosmos-RL

pip install "git+https://github.com/nvidia-cosmos/cosmos-rl.git"

# Download a native SANA checkpoint

python -m tools.download find_model \
    --model_path SANA_LinearFFN.pth \
    --model_source hf://yitongl/SANA_LinearFFN/SANA_LinearFFN.pth

Post-Training Workflows

Cosmos-RL for post-training supports both supervised fine-tuning and reinforcement learning through unified configuration files under configs/sol_rl/. The default configuration in configs/sol_rl/sana.py defines resolution settings (1024px), LoRA target modules, and rollout parameters.

Supervised Fine-Tuning with LoRA

For parameter-efficient supervised fine-tuning, Cosmos-RL wraps the SANA transformer in a PEFT LoRA adapter. The configuration sets preview_model and fullrollout_model to PEFT mode while maintaining the base checkpoint frozen, enabling efficient updates with minimal memory overhead.

cosmos-rl \
  --config ./configs/sol_rl/sana-image-sft-lora.toml \
  cosmos_rl.tools.dataset.diffusers_dataset

Reinforcement Learning with Diffusion-NFT

The Diffusion-NFT algorithm executes rollouts using pipeline_with_logprob_sana from diffusion/post_training/diffusers_patch/pipeline_with_logprob.py to generate samples and compute log-probabilities for selection. The preset function sana_diffusionnft_pickscore creates ready-to-run configs that set fullrollout_model to PEFT mode and sample.full_rollout_num to best-of-n sampling.

cosmos-rl \
  --config ./configs/sol_rl/sana-image-nft.toml \
  cosmos_rl.tools.dataset.diffusion_nft

Video Post-Training with OCR Rewards

Video post-training reuses the same rollout logic but processes 3D latents through the sampler. The dataset loader returns video frames (e.g., .mp4 files with OCR annotations), and the pipeline handles temporal dimensions without code changes to the underlying transformer.

cosmos-rl \
  --config ./configs/sol_rl/sana-video-nft.toml \
  cosmos_rl.tools.dataset.diffusion_nft

Optimizing Rollout Parameters

The training script includes automatic batch size selection to optimize GPU memory usage during rollouts. Use the _auto_rollout_bsz helper to calculate compatible batch sizes that evenly divide your total images per prompt, ensuring efficient parallel generation.

from train_scripts.sol_rl.train_sana import _auto_rollout_bsz

# Calculate batch size for 48 images per prompt

rollout_bsz = _auto_rollout_bsz(num_image_per_prompt=48)  # Returns 12

Monitoring and Logging

Training metrics and generated samples are logged through WandB and the log_rollout_images helper. On-disk logs are stored under logs/ with subdirectories organized by experiment configuration, while the asynchronous reward services report scores independently of the main training loop.

Summary

• Cosmos-RL for post-training SANA provides a unified framework for both supervised fine-tuning and reinforcement learning on image and video models through train_scripts/sol_rl/train_sana.py.
• The integration supports PEFT LoRA adapters and compilation modes including NVFP4 for accelerated inference, configured via configs/sol_rl/sana.py.
• Reward services operate asynchronously, supporting PickScore, CLIPScore, ImageReward, and HPSv2 without blocking the training loop.
• The same codebase handles 2D images and 3D videos through diffusion/model/nets/sana_multi_scale.py and the pipeline_with_logprob_sana sampler.
• Native checkpoint loading via _resolve_native_checkpoint_source ensures compatibility with original SANA model weights.

Frequently Asked Questions

What reward functions does Cosmos-RL support for SANA post-training?

Cosmos-RL supports asynchronous scoring through PickScore, CLIPScore, ImageReward, and HPSv2. These reward services run in separate processes, querying generated samples via the rollout pipeline and returning gradients or rankings to the RL algorithms without interrupting the main training loop.

Can I use Cosmos-RL with both SANA image and video models?

Yes. The underlying architecture in diffusion/model/nets/sana_multi_scale.py handles both 2D image and 3D video latents. You use the same train_scripts/sol_rl/train_sana.py script for both modalities, changing only the dataset configuration to point to image directories or video files (.mp4), while the sampler automatically adapts to the latent dimensions.

How does the PEFT LoRA integration work in the training pipeline?

The pipeline wraps the SANA transformer in a PEFT LoRA adapter when preview_model or fullrollout_model is set to "peft" in the configuration. This occurs after loading the native checkpoint via _resolve_native_checkpoint_source, allowing efficient fine-tuning with minimal memory overhead while keeping the base model frozen during rollouts.

Where are the rollout logs and metrics stored during training?

Training metrics and generated samples are logged to WandB and saved on disk under the logs/ directory. The log_rollout_images helper in the training script saves visualizations of rollouts for inspection, while experiment-specific configurations determine the exact subdirectory structure based on the algorithm and reward function used.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}