# How Pixelle-Video Integrates with ComfyUI for Video Generation: A Complete Technical Guide

> Learn how Pixelle-Video integrates with ComfyUI using ComfyKit for seamless video generation. This guide details connecting to self-hosted or cloud ComfyUI servers for efficient workflow execution.

- Repository: [AIDC-AI/Pixelle-Video](https://github.com/AIDC-AI/Pixelle-Video)
- Tags: how-to-guide
- Published: 2026-04-23

---

**Pixelle-Video integrates with ComfyUI through ComfyKit, a Python client library that manages lazy-initialized connections to either self-hosted ComfyUI servers or RunningHub cloud services, enabling seamless video generation via workflow execution.**

The **AIDC-AI/Pixelle-Video** repository provides a production-ready abstraction layer that transforms ComfyUI's node-based workflow system into programmatic API calls. This design lets developers generate videos from Python code or web interfaces without manually constructing complex JSON workflows.

## Core Architecture: The ComfyKit Client Pattern

The integration centers on a **shared, lazily-initialized `ComfyKit` instance** managed by `PixelleVideoCore`. This pattern ensures efficient resource usage while supporting configuration hot-reloading.

### Configuration Layer

User settings for ComfyUI connectivity reside in [`config.yaml`](https://github.com/AIDC-AI/Pixelle-Video/blob/main/config.yaml) and are parsed by [`pixelle_video/config/manager.py`](https://github.com/AIDC-AI/Pixelle-Video/blob/main/pixelle_video/config/manager.py). The core extracts these values through `PixelleVideoCore._get_comfykit_config()` ([source](https://github.com/AIDC-AI/Pixelle-Video/blob/main/pixelle_video/service.py#L14-L27)):

```python

# config.yaml structure

comfyui_url: "http://localhost:8188"
runninghub_api_key: "your-api-key"
runninghub_secret: "your-secret"
default_workflow: "video_film.json"

```

### Lazy Initialization with Hot-Reload

The `PixelleVideoCore._get_or_create_comfykit()` method ([source](https://github.com/AIDC-AI/Pixelle-Video/blob/main/pixelle_video/service.py#L44-L76)) implements intelligent instance management:

1. **First call**: Creates `ComfyKit` with current config, caches config hash
2. **Subsequent calls**: Returns cached instance if config unchanged
3. **Config change detected**: Invalidates cache, creates fresh instance

This enables zero-downtime configuration updates without restating the entire application.

## Service Layer: ComfyBaseService Abstraction

All ComfyUI-backed services inherit from `ComfyBaseService` in [`pixelle_video/services/comfy_base_service.py`](https://github.com/AIDC-AI/Pixelle-Video/blob/main/pixelle_video/services/comfy_base_service.py) ([source](https://github.com/AIDC-AI/Pixelle-Video/blob/main/pixelle_video/services/comfy_base_service.py#L32-L45)). This base class provides:

- **Workflow scanning**: Discovers available workflows in `workflows/selfhost/` and `workflows/runninghub/`
- **Workflow selection**: Resolves user-provided names to workflow info dictionaries
- **Config building**: Constructs `ComfyKit` configuration for both self-hosted and RunningHub sources

### Workflow Resolution Logic

The `_resolve_workflow()` method ([source](https://github.com/AIDC-AI/Pixelle-Video/blob/main/pixelle_video/services/comfy_base_service.py#L87-L106)) handles three resolution paths:

| Source | Resolution Behavior |
|--------|---------------------|
| `selfhost` | Returns local file path to JSON workflow |
| `runninghub` | Returns remote `workflow_id` with API credentials |
| Default | Uses configured default workflow from [`config.yaml`](https://github.com/AIDC-AI/Pixelle-Video/blob/main/config.yaml) |

## Video Generation: MediaService Implementation

The `MediaService` class in [`pixelle_video/services/media.py`](https://github.com/AIDC-AI/Pixelle-Video/blob/main/pixelle_video/services/media.py) provides the primary programmatic interface for video generation. It extends `ComfyBaseService` with video-specific parameter handling.

### Parameter Building

The service constructs workflow parameters that can include duration matching for TTS audio synchronization ([source](https://github.com/AIDC-AI/Pixelle-Video/blob/main/pixelle_video/services/media.py#L1-L30)):

```python

# MediaService parameter construction

params = {
    "prompt": user_prompt,
    "width": width,
    "height": height,
    "steps": steps,
    "cfg": cfg_scale,
    "duration": duration_seconds,  # Optional: for audio sync

}

```

### Execution Flow

The `media()` method ([source](https://github.com/AIDC-AI/Pixelle-Video/blob/main/pixelle_video/services/media.py#L28-L43)) executes as follows:

1. Resolves workflow via base class
2. Obtains shared `ComfyKit` from `PixelleVideoCore`
3. Calls `kit.execute(workflow_input, workflow_params)`
4. Handles both self-hosted and RunningHub response formats

### Result Processing

The service extracts video URLs from the execution result ([source](https://github.com/AIDC-AI/Pixelle-Video/blob/main/pixelle_video/services/media.py#L52-L71)):

```python

# Result extraction logic

if result.videos:
    video_url = result.videos[0].url
elif result.outputs and "videos" in result.outputs:
    video_url = result.outputs["videos"][0]
    
return MediaResult(url=video_url, duration=duration)

```

## UI Integration: The i2v Pipeline

The Streamlit-based web interface demonstrates direct `ComfyKit` usage in [`web/pipelines/i2v.py`](https://github.com/AIDC-AI/Pixelle-Video/blob/main/web/pipelines/i2v.py) ([source](https://github.com/AIDC-AI/Pixelle-Video/blob/main/web/pipelines/i2v.py#L102-L105)).

### Pipeline Execution

The image-to-video pipeline obtains the shared client:

```python

# UI pipeline accessing shared ComfyKit

kit = await pixelle_video._get_or_create_comfykit()

```

It then builds parameters and decides between local file paths or RunningHub workflow IDs ([source](https://github.com/AIDC-AI/Pixelle-Video/blob/main/web/pipelines/i2v.py#L28-L33)):

```python

# Parameter building with source detection

workflow_params = {
    "image": image_path,
    "prompt": prompt,
}

# workflow_input varies by source (local path vs. remote ID)

```

### Result Download and Display

The pipeline executes the workflow, downloads the resulting video, and presents it in the UI ([source](https://github.com/AIDC-AI/Pixelle-Video/blob/main/web/pipelines/i2v.py#L33-L45)):

```python

# Execution and result handling

result = await kit.execute(workflow_input, workflow_params)

# Extract and download first video

video_url = result.videos[0].url
local_path = download_video(video_url)
display_video(local_path)

```

## Code Examples

### Complete Self-Hosted Workflow

```python
import asyncio
from pixelle_video import pixelle_video

async def generate_local_video():
    """Generate video using self-hosted ComfyUI instance."""
    await pixelle_video.initialize()
    
    result = await pixelle_video.media(
        prompt="A futuristic cityscape at sunset with flying vehicles",
        workflow="video_film.json",
        media_type="video",
        width=1280,
        height=720,
        duration=12.0,  # Match TTS audio length

        steps=30,
        cfg=7.0,
    )
    
    print(f"Video ready: {result.url}")
    print(f"Duration: {result.duration}s")
    
    await pixelle_video.cleanup()

asyncio.run(generate_local_video())

```

### Cloud-Based RunningHub Workflow

```python
import asyncio
from pixelle_video import pixelle_video

async def generate_cloud_video():
    """Generate video using RunningHub cloud service."""
    await pixelle_video.initialize()
    
    # Workflow prefix triggers RunningHub resolution

    result = await pixelle_video.media(
        prompt="A dragon flying over misty mountains",
        workflow="runninghub/video_film.json",
        media_type="video",
        duration=8.5,
    )
    
    print(f"Cloud video URL: {result.url}")
    
    await pixelle_video.cleanup()

asyncio.run(generate_cloud_video())

```

## Summary

Pixelle-Video's ComfyUI integration delivers these key capabilities:

- **Lazy-initialized shared client** via `PixelleVideoCore._get_or_create_comfykit()` eliminates redundant connections and supports configuration hot-reloading
- **Dual-source workflow support** handles both local JSON files (`selfhost`) and RunningHub cloud workflows (`runninghub`) through unified resolution logic
- **Service-oriented architecture** with `ComfyBaseService` providing common workflow scanning, selection, and execution patterns
- **Direct UI pipeline access** in [`web/pipelines/i2v.py`](https://github.com/AIDC-AI/Pixelle-Video/blob/main/web/pipelines/i2v.py) demonstrating immediate `ComfyKit` usage for image-to-video generation
- **Programmatic API** via `MediaService.media()` with built-in duration matching for audio-visual synchronization

## Frequently Asked Questions

### What is ComfyKit and how does it connect to ComfyUI?

**ComfyKit is a Python client library that provides asynchronous communication with ComfyUI servers.** It handles workflow submission, progress polling, and result retrieval through both HTTP API (self-hosted) and RunningHub's managed service. Pixelle-Video wraps ComfyKit with lazy initialization and configuration management to simplify integration across multiple services.

### Can I use Pixelle-Video with a local ComfyUI installation?

**Yes, self-hosted ComfyUI is fully supported.** Configure `comfyui_url` in your [`config.yaml`](https://github.com/AIDC-AI/Pixelle-Video/blob/main/config.yaml) to point to your local instance (typically `http://localhost:8188`). Place workflow JSON files in `workflows/selfhost/` and reference them by name. The `MediaService` automatically detects local workflows and uses file-based execution rather than cloud API calls.

### How does workflow resolution differ between self-hosted and RunningHub sources?

**Self-hosted workflows resolve to local file paths**, while **RunningHub workflows resolve to remote `workflow_id` strings with associated credentials.** The `ComfyBaseService._resolve_workflow()` method examines the workflow name prefix: names starting with `runninghub/` trigger cloud resolution using configured API keys, while plain names resolve against the local `workflows/` directory. Both paths produce a unified `workflow_info` dictionary consumed by `ComfyKit.execute()`.

### What happens when I change ComfyUI configuration while Pixelle-Video is running?

**Configuration changes trigger automatic ComfyKit re-initialization.** The `PixelleVideoCore` hashes the current configuration on each `_get_or_create_comfykit()` call. If the hash differs from the cached instance's hash, the old `ComfyKit` is closed and a new instance created with updated settings. This enables hot-reloading of endpoints, credentials, or default workflows without application restarts.