# How ViMax Ensures Character Consistency Across Multiple Video Shots

> ViMax ensures character consistency by binding visual elements to a single portrait set for reuse across video shots. Learn how ViMax maintains character continuity in video generation.

- Repository: [✨Data Intelligence Lab@HKU✨/ViMax](https://github.com/HKUDS/ViMax)
- Tags: internals
- Published: 2026-05-20

---

**ViMax guarantees character consistency by binding every visual element to a single, reusable portrait set generated once per character and referenced throughout the entire video-generation pipeline.**

Maintaining visual continuity in AI-generated video remains one of the hardest challenges in generative media. The HKUDS/ViMax repository solves this through a strict registry-based architecture that eliminates character drift by reusing identical reference images across every shot.

## The Three-Stage Consistency Pipeline

ViMax implements character consistency across multiple video shots through three tightly coupled stages:

1. **Character extraction** – Parses the script and creates a `CharacterInScene` model describing each character’s static and dynamic traits.
2. **Portrait generation** – Builds front, side, and back portraits once per character and stores them in a shared registry.
3. **Frame rendering** – Every shot pulls appropriate portraits from the registry as reference images, ensuring the same pixel data drives every frame.

## Unified Character Schema and Registry

At the heart of the system lies the **unified character schema** defined in [`interfaces/character.py`](https://github.com/HKUDS/ViMax/blob/main/interfaces/character.py). This `CharacterInScene` model serves as the single source of truth for every downstream component.

```python
class CharacterInScene(BaseModel):
    idx: int
    identifier_in_scene: str               # e.g. "Alice"

    is_visible: bool
    static_features: str                   # immutable traits (hair, eyes, build)

    dynamic_features: Optional[str] = None  # clothing, accessories

```

The `CharacterExtractor` agent (located in [`main/agents/character_extractor.py`](https://github.com/HKUDS/ViMax/blob/main/main/agents/character_extractor.py)) parses input scripts and instantiates these models, capturing immutable traits like hair color and facial structure separately from dynamic elements like clothing. This separation ensures that static visual identifiers remain constant while allowing contextual variation in attire.

## One-Time Portrait Generation

After extraction, the `Script2VideoPipeline` checks the [`character_portraits_registry.json`](https://github.com/HKUDS/ViMax/blob/main/character_portraits_registry.json) file. If a character is missing from this registry, the pipeline invokes `generate_portraits_for_single_character` to create three canonical views: **front**, **side**, and **back**.

As implemented in [`main/pipelines/script2video_pipeline.py`](https://github.com/HKUDS/ViMax/blob/main/main/pipelines/script2video_pipeline.py), the generation produces a nested dictionary structure:

```json
{
  "Alice": {
    "front": {"path": ".../front.png", "description": "A front view portrait of Alice."},
    "side":  {"path": ".../side.png",  "description": "A side view portrait of Alice."},
    "back":  {"path": ".../back.png",  "description": "A back view portrait of Alice."}
  }
}

```

These portraits are written once per character and reused for the entire pipeline run. The registry locks visual identity to specific file paths, ensuring that subsequent frames reference identical pixel data regardless of shot changes or camera movements.

## Reference Image Injection for Every Frame

During frame generation, ViMax pulls consistent portraits from the registry for every shot type—whether generating first frames, last frames, or transition frames. The `generate_frame_for_single_shot` function constructs `available_image_path_and_text_pairs` by looking up visible characters in the registry:

```python
for character_idx in shot_descriptions[first_shot_idx].ff_vis_char_idxs:
    identifier = characters[character_idx].identifier_in_scene
    registry_item = character_portraits_registry[identifier]
    for view, item in registry_item.items():
        available_image_path_and_text_pairs.append((item["path"], item["description"]))

```

Because the same file paths from `character_portraits_registry` feed into the image-generation model for every shot featuring that character, the model receives stable visual cues. This prevents the subtle drift in facial features or body proportions that typically occurs when AI video generators recreate characters from text descriptions alone.

## Event-Driven Synchronization

To eliminate race conditions between portrait creation and frame consumption, ViMax implements an **event-driven synchronization** mechanism. Each character has an associated `asyncio.Event` stored in `character_portrait_events[character.idx]`.

Downstream frame-generation tasks await this event before executing, ensuring portrait files are fully written to disk before any shot attempts to consume them. As seen in [`main/pipelines/script2video_pipeline.py`](https://github.com/HKUDS/ViMax/blob/main/main/pipelines/script2video_pipeline.py), this coordination prevents scenarios where a frame generator might read partially written or corrupted portrait data.

## Summary

- **Single registry architecture**: ViMax stores canonical portraits in [`character_portraits_registry.json`](https://github.com/HKUDS/ViMax/blob/main/character_portraits_registry.json) and references them by `identifier_in_scene` throughout generation.
- **Three-view consistency**: Each character receives front, side, and back portraits generated once by `generate_portraits_for_single_character`.
- **Path-based reference**: Frame generation functions like `generate_frame_for_single_shot` inject exact file paths from the registry, ensuring pixel-perfect consistency across shots.
- **Async safety**: `asyncio.Event` objects synchronize portrait generation with frame rendering, preventing race conditions.

## Frequently Asked Questions

### What is character consistency in video generation?

Character consistency refers to maintaining stable visual attributes—such as facial features, body type, and clothing style—across multiple shots or scenes in a generated video. Without explicit mechanisms to preserve identity, AI models tend to drift, producing characters with varying appearances between cuts.

### How does ViMax prevent character drift between shots?

ViMax prevents drift by generating a fixed set of portrait images **once** per character and storing them in [`character_portraits_registry.json`](https://github.com/HKUDS/ViMax/blob/main/character_portraits_registry.json). Every subsequent shot pulls these exact image files as reference material for the image-generation model. Because the model receives identical pixel data for "Alice" in shot one and shot ten, the character's appearance remains constant.

### What are the three portrait views generated for each character?

The system generates **front**, **side**, and **back** views for every character via `generate_portraits_for_single_character`. These multiple angles allow the rendering pipeline to select the most appropriate reference based on camera positioning while maintaining geometric consistency in the character's proportions and features.

### How does ViMax handle synchronization between portrait generation and frame rendering?

ViMax uses `asyncio.Event` objects mapped to each character index. Frame generation tasks await `character_portrait_events[character.idx]` before executing, ensuring that portrait files are completely written and ready before any shot attempts to load them. This prevents race conditions where a frame might reference an incomplete or missing portrait file.