How Pixelle-Video Integrates with ComfyUI for Video Generation: A Complete Technical Guide

Pixelle-Video integrates with ComfyUI through ComfyKit, a Python client library that manages lazy-initialized connections to either self-hosted ComfyUI servers or RunningHub cloud services, enabling seamless video generation via workflow execution.

The AIDC-AI/Pixelle-Video repository provides a production-ready abstraction layer that transforms ComfyUI's node-based workflow system into programmatic API calls. This design lets developers generate videos from Python code or web interfaces without manually constructing complex JSON workflows.

Core Architecture: The ComfyKit Client Pattern

The integration centers on a shared, lazily-initialized ComfyKit instance managed by PixelleVideoCore. This pattern ensures efficient resource usage while supporting configuration hot-reloading.

Configuration Layer

User settings for ComfyUI connectivity reside in config.yaml and are parsed by pixelle_video/config/manager.py. The core extracts these values through PixelleVideoCore._get_comfykit_config() (source):

# config.yaml structure

comfyui_url: "http://localhost:8188"
runninghub_api_key: "your-api-key"
runninghub_secret: "your-secret"
default_workflow: "video_film.json"

Lazy Initialization with Hot-Reload

The PixelleVideoCore._get_or_create_comfykit() method (source) implements intelligent instance management:

1. First call: Creates ComfyKit with current config, caches config hash
2. Subsequent calls: Returns cached instance if config unchanged
3. Config change detected: Invalidates cache, creates fresh instance

This enables zero-downtime configuration updates without restating the entire application.

Service Layer: ComfyBaseService Abstraction

All ComfyUI-backed services inherit from ComfyBaseService in pixelle_video/services/comfy_base_service.py (source). This base class provides:

• Workflow scanning: Discovers available workflows in workflows/selfhost/ and workflows/runninghub/
• Workflow selection: Resolves user-provided names to workflow info dictionaries
• Config building: Constructs ComfyKit configuration for both self-hosted and RunningHub sources

Workflow Resolution Logic

The _resolve_workflow() method (source) handles three resolution paths:

Source	Resolution Behavior
selfhost	Returns local file path to JSON workflow
runninghub	Returns remote workflow_id with API credentials
Default	Uses configured default workflow from config.yaml

Video Generation: MediaService Implementation

The MediaService class in pixelle_video/services/media.py provides the primary programmatic interface for video generation. It extends ComfyBaseService with video-specific parameter handling.

Parameter Building

The service constructs workflow parameters that can include duration matching for TTS audio synchronization (source):

# MediaService parameter construction

params = {
    "prompt": user_prompt,
    "width": width,
    "height": height,
    "steps": steps,
    "cfg": cfg_scale,
    "duration": duration_seconds,  # Optional: for audio sync

}

Execution Flow

The media() method (source) executes as follows:

1. Resolves workflow via base class
2. Obtains shared ComfyKit from PixelleVideoCore
3. Calls kit.execute(workflow_input, workflow_params)
4. Handles both self-hosted and RunningHub response formats

Result Processing

The service extracts video URLs from the execution result (source):

# Result extraction logic

if result.videos:
    video_url = result.videos[0].url
elif result.outputs and "videos" in result.outputs:
    video_url = result.outputs["videos"][0]
    
return MediaResult(url=video_url, duration=duration)

UI Integration: The i2v Pipeline

The Streamlit-based web interface demonstrates direct ComfyKit usage in web/pipelines/i2v.py (source).

Pipeline Execution

The image-to-video pipeline obtains the shared client:

# UI pipeline accessing shared ComfyKit

kit = await pixelle_video._get_or_create_comfykit()

It then builds parameters and decides between local file paths or RunningHub workflow IDs (source):

# Parameter building with source detection

workflow_params = {
    "image": image_path,
    "prompt": prompt,
}

# workflow_input varies by source (local path vs. remote ID)

Result Download and Display

The pipeline executes the workflow, downloads the resulting video, and presents it in the UI (source):

# Execution and result handling

result = await kit.execute(workflow_input, workflow_params)

# Extract and download first video

video_url = result.videos[0].url
local_path = download_video(video_url)
display_video(local_path)

Code Examples

Complete Self-Hosted Workflow

import asyncio
from pixelle_video import pixelle_video

async def generate_local_video():
    """Generate video using self-hosted ComfyUI instance."""
    await pixelle_video.initialize()
    
    result = await pixelle_video.media(
        prompt="A futuristic cityscape at sunset with flying vehicles",
        workflow="video_film.json",
        media_type="video",
        width=1280,
        height=720,
        duration=12.0,  # Match TTS audio length

        steps=30,
        cfg=7.0,
    )
    
    print(f"Video ready: {result.url}")
    print(f"Duration: {result.duration}s")
    
    await pixelle_video.cleanup()

asyncio.run(generate_local_video())

Cloud-Based RunningHub Workflow

import asyncio
from pixelle_video import pixelle_video

async def generate_cloud_video():
    """Generate video using RunningHub cloud service."""
    await pixelle_video.initialize()
    
    # Workflow prefix triggers RunningHub resolution

    result = await pixelle_video.media(
        prompt="A dragon flying over misty mountains",
        workflow="runninghub/video_film.json",
        media_type="video",
        duration=8.5,
    )
    
    print(f"Cloud video URL: {result.url}")
    
    await pixelle_video.cleanup()

asyncio.run(generate_cloud_video())

Summary

Pixelle-Video's ComfyUI integration delivers these key capabilities:

• Lazy-initialized shared client via PixelleVideoCore._get_or_create_comfykit() eliminates redundant connections and supports configuration hot-reloading
• Dual-source workflow support handles both local JSON files (selfhost) and RunningHub cloud workflows (runninghub) through unified resolution logic
• Service-oriented architecture with ComfyBaseService providing common workflow scanning, selection, and execution patterns
• Direct UI pipeline access in web/pipelines/i2v.py demonstrating immediate ComfyKit usage for image-to-video generation
• Programmatic API via MediaService.media() with built-in duration matching for audio-visual synchronization

Frequently Asked Questions

What is ComfyKit and how does it connect to ComfyUI?

ComfyKit is a Python client library that provides asynchronous communication with ComfyUI servers. It handles workflow submission, progress polling, and result retrieval through both HTTP API (self-hosted) and RunningHub's managed service. Pixelle-Video wraps ComfyKit with lazy initialization and configuration management to simplify integration across multiple services.

Can I use Pixelle-Video with a local ComfyUI installation?

Yes, self-hosted ComfyUI is fully supported. Configure comfyui_url in your config.yaml to point to your local instance (typically http://localhost:8188). Place workflow JSON files in workflows/selfhost/ and reference them by name. The MediaService automatically detects local workflows and uses file-based execution rather than cloud API calls.

How does workflow resolution differ between self-hosted and RunningHub sources?

Self-hosted workflows resolve to local file paths, while RunningHub workflows resolve to remote workflow_id strings with associated credentials. The ComfyBaseService._resolve_workflow() method examines the workflow name prefix: names starting with runninghub/ trigger cloud resolution using configured API keys, while plain names resolve against the local workflows/ directory. Both paths produce a unified workflow_info dictionary consumed by ComfyKit.execute().

What happens when I change ComfyUI configuration while Pixelle-Video is running?

Configuration changes trigger automatic ComfyKit re-initialization. The PixelleVideoCore hashes the current configuration on each _get_or_create_comfykit() call. If the hash differs from the cached instance's hash, the old ComfyKit is closed and a new instance created with updated settings. This enables hot-reloading of endpoints, credentials, or default workflows without application restarts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}