How to Set Up Local GPU Video Generation with WAN, Hunyuan, CogVideo, and LTX-Video in OpenMontage

Enable local generation by setting VIDEO_GEN_LOCAL_ENABLED=true, install the Diffusers stack via pip, and instantiate one of the four provider classes—WanVideo, HunyuanVideo, CogVideoVideo, or LTXVideoLocal—to generate videos entirely on your GPU.

OpenMontage provides a unified tool abstraction for local video generation that supports multiple state-of-the-art diffusion models. By leveraging the shared plumbing in tools/video/_shared.py, you can run WAN, Hunyuan, CogVideo, and LTX-Video inference directly on NVIDIA GPUs or Apple Silicon without external API calls. This guide walks you through the exact setup steps, environment configuration, and code implementation required to activate local GPU video generation in the OpenMontage framework.

Prerequisites and Hardware Requirements

OpenMontage supports four distinct local providers, each with specific VRAM requirements:

Provider	Tool Class	Default Model Variant	VRAM Requirement
WAN	WanVideo	wan2.2-ti2v-5b	12 GB
Hunyuan	HunyuanVideo	hunyuan-1.5	14 GB
CogVideo	CogVideoVideo	cogvideo-5b	12 GB
LTX-Video	LTXVideoLocal	ltx2-local	12 GB

All four tools inherit from the base tool class defined in tools/base_tool.py and utilize the shared generation logic in tools/video/_shared.py.

Environment Setup and Installation

Enable Local Generation Mode

Before importing any video tools, you must explicitly enable local generation via environment variable. When VIDEO_GEN_LOCAL_ENABLED is not set to a truthy value, all video tools return ToolStatus.UNAVAILABLE and raise informative errors prompting you to complete setup.

Export the toggle in your shell:

export VIDEO_GEN_LOCAL_ENABLED=true

Install the Diffusers Stack

Each tool relies on the Hugging Face Diffusers ecosystem. The helper function local_install_instructions() in tools/video/_shared.py generates the exact installation command. Run the following to install all required dependencies:

uv pip install diffusers transformers accelerate torch pillow requests

Note: While the example uses uv, standard pip works identically.

Verify GPU Visibility

Confirm PyTorch can detect your hardware before running generation:

import torch

print(torch.cuda.is_available())  # True for NVIDIA GPUs

print(torch.backends.mps.is_available())  # True for Apple Silicon

The get_torch_device() function in tools/video/_shared.py automatically selects the best available device: CUDA is preferred, with fallback to MPS (Apple Silicon), then CPU.

The Shared Local Generation Pipeline

All four providers call the centralized generate_local_video() function defined in tools/video/_shared.py. This function orchestrates the complete inference workflow:

1. Variant Resolution – Maps the requested model variant to its metadata dictionary
2. Pipeline Loading – Invokes load_diffusers_pipeline() to initialize the correct Diffusers pipeline class
3. Argument Construction – Builds generation arguments including prompt, width, height, num_frames, and reference images
4. Precision Handling – Automatically selects bfloat16 for CUDA (when supported), float16 for MPS, and float32 for CPU
5. Model Off-Loading – Calls pipeline.enable_model_cpu_offload() when enable_model_offload=True and the device is CUDA
6. Video Export – Exports frames to MP4 format and returns a ToolResult containing the output path

The function signature is:

def generate_local_video(*, tool_name, variants, default_variant, inputs) -> ToolResult:
    # Implementation in tools/video/_shared.py

Generating Videos with Each Provider

WAN Video Generation

The WanVideo class in tools/video/wan_video.py implements the WAN 2.2 architecture. It uses the low-level engine in tools/video/_wan_engine.py for segment planning and memory optimization.

from tools.video.wan_video import WanVideo

tool = WanVideo()

result = tool.execute({
    "prompt": "A futuristic city skyline at sunrise, cinematic lighting",
    "model_variant": "wan2.2-ti2v-5b",  # Optional: defaults to 5B variant

    "width": 1280,
    "height": 704,
    "num_frames": 121,
    "enable_model_offload": True,  # Reduces VRAM usage via CPU off-loading

    "output_path": "wan_output.mp4"
})

if result.success:
    print(f"Video saved to: {result.data['output']}")

HunyuanVideo Generation

The HunyuanVideo class in tools/video/hunyuan_video.py provides access to the Hunyuan 1.5 model.

from tools.video.hunyuan_video import HunyuanVideo

tool = HunyuanVideo()

result = tool.execute({
    "prompt": "A magical forest with floating lanterns, ultra-detailed",
    "model_variant": "hunyuan-1.5",
    "output_path": "hunyuan_output.mp4"
})

CogVideo Generation

The CogVideoVideo class in tools/video/cogvideo_video.py handles CogVideo 5B inference with per-variant capability checks.

from tools.video.cogvideo_video import CogVideoVideo

tool = CogVideoVideo()

result = tool.execute({
    "prompt": "An astronaut riding a horse through space",
    "model_variant": "cogvideo-5b",
    "output_path": "cogvideo_output.mp4"
})

LTX-Video Local Generation

The LTXVideoLocal class in tools/video/ltx_video_local.py runs the Lightweight Video Transformer locally.

from tools.video.ltx_video_local import LTXVideoLocal

tool = LTXVideoLocal()

result = tool.execute({
    "prompt": "Ocean waves crashing against rocky cliffs",
    "model_variant": "ltx2-local",
    "output_path": "ltx_output.mp4"
})

Image-to-Video and Advanced Workflows

All providers support image-to-video generation through the load_reference_image() function in tools/video/_shared.py. Pass either a local path or remote URL via the reference_image_path or reference_image_url keys:

result = tool.execute({
    "prompt": "Turn the sketch into a looping animation",
    "operation": "image_to_video",
    "reference_image_path": "sketch.png",
    "width": 1024,
    "height": 576,
    "output_path": "sketch_animation.mp4"
})

The system automatically resizes the reference image to match your generation dimensions and injects it into the pipeline's image conditioning.

Memory Optimization Strategies

To run these models on GPUs with limited VRAM, utilize the model off-loading feature:

result = tool.execute({
    "prompt": "Complex scene with many details",
    "enable_model_offload": True,  # Keeps inactive weights on CPU

    "output_path": "optimized.mp4"
})

When enable_model_offload is enabled and CUDA is available, the pipeline executes enable_model_cpu_offload(), significantly reducing peak VRAM usage at the cost of marginal speed reduction during model layer transitions.

Summary

• Enable local mode by exporting VIDEO_GEN_LOCAL_ENABLED=true before running any code.
• Install dependencies using the command provided by local_install_instructions() in tools/video/_shared.py.
• Choose your provider by importing WanVideo, HunyuanVideo, CogVideoVideo, or LTXVideoLocal from their respective modules in tools/video/.
• Manage VRAM by setting enable_model_offload=True for 12GB cards, or run natively on 14GB+ cards.
• Generate videos by passing a payload dictionary containing at minimum a prompt key and output_path.
• Extend to image-to-video by including reference_image_path or reference_image_url in your inputs.

Frequently Asked Questions

What is the minimum GPU requirement for local video generation in OpenMontage?

You need a GPU with at least 12 GB of VRAM to run the default model variants for WAN, CogVideo, and LTX-Video. HunyuanVideo requires 14 GB VRAM. The system automatically handles precision selection (bfloat16/float16) based on your hardware to optimize memory usage.

How does OpenMontage handle device selection when multiple GPUs are available?

The get_torch_device() function in tools/video/_shared.py checks for CUDA availability first, then falls back to Apple Silicon MPS, and finally CPU. It does not currently implement multi-GPU sharding; the model loads onto the default CUDA device or the specified MPS/CPU device.

Why does my video tool return ToolStatus.UNAVAILABLE?

This status indicates either the VIDEO_GEN_LOCAL_ENABLED environment variable is not set to a truthy value, or the required Python packages (diffusers, transformers, accelerate) are missing from your environment. Check the error message for the exact pip install command generated by local_install_instructions().

Can I use OpenMontage for image-to-video generation with these local models?

Yes. Pass the operation: "image_to_video" key in your payload along with either reference_image_path (local file) or reference_image_url (HTTP URL). The load_reference_image() helper in tools/video/_shared.py handles fetching, decoding, and resizing the reference image before passing it to the Diffusers pipeline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}