ViMax Character Portrait Registry: How Front, Side, and Back Views Are Generated

ViMax's character portrait registry is a JSON-backed dictionary that stores pre-rendered front, side, and back portrait images for every character, along with textual descriptions used during video generation.

The HKUDS/ViMax open-source repository implements a sophisticated character management system that ensures visual consistency across video scenes. The character portrait registry acts as a centralized cache, eliminating redundant image generation while providing reference materials for consistent character depiction from multiple angles.

What is the ViMax Character Portrait Registry?

The registry functions as a persistent data structure that maps character identifiers to their corresponding portrait assets. According to the source code in pipelines/script2video_pipeline.py, the system creates this registry once per run and persists it to character_portraits_registry.json to enable reuse across sessions.

Registry Structure and Persistence

Each entry in the registry uses the character's identifier_in_scene (e.g., "hero") as the top-level key. The JSON structure stores three distinct views:

{
  "CharacterA": {
    "front": { 
      "path": ".../front.png",  
      "description": "A front view portrait of CharacterA." 
    },
    "side": { 
      "path": ".../side.png",   
      "description": "A side view portrait of CharacterA." 
    },
    "back": { 
      "path": ".../back.png",   
      "description": "A back view portrait of CharacterA." 
    }
  }
}

Each view object contains:

• path: Absolute file path to the generated PNG
• description: A short human-readable description used for reference-image selection during frame generation

How ViMax Generates Front, Side, and Back Views

The CharacterPortraitsGenerator class in agents/character_portraits_generator.py implements a sequential generation strategy that uses the front view as an identity anchor for subsequent angles.

Front View Generation

The front view serves as the foundation for character identity. The generate_front_portrait method constructs a generation prompt using prompt_template_front and invokes the image generation agent:

• Location: agents/character_portraits_generator.py lines 17-24 and 44-55
• Approach: Generates directly from the character's static and dynamic feature strings
• Output: Base identity image used for subsequent view generation

Side View Generation

The side view generation maintains character consistency by referencing the existing front portrait. The generate_side_portrait method explicitly passes the front image as a reference:

# From agents/character_portraits_generator.py lines 61-73

reference_image_paths=[front_image_path]

• Template: Uses prompt_template_side to guide angle transformation
• Reference Strategy: Supplies front_image_path as reference_image_paths to preserve identity features while rotating the perspective 90 degrees

Back View Generation

Similarly, the back view reuses the front portrait as a reference but modifies the prompt to indicate facial feature occlusion:

• Method: generate_back_portrait (lines 78-90 in agents/character_portraits_generator.py)
• Template: Uses prompt_template_back with instructions to hide facial features
• Consistency: Maintains hair, clothing, and physical proportions via the same reference_image_paths mechanism

Retry Logic and Error Handling

All three generation methods are wrapped with a tenacity retry decorator to handle transient AI-service failures:

@retry(stop=stop_after_attempt(3), ...)

This ensures pipeline robustness when calling external image generation APIs.

Orchestration Flow in the Pipeline

The Script2VideoPipeline coordinates registry creation through the generate_character_portraits method (lines 48-73 in pipelines/script2video_pipeline.py).

Initialization Check

The pipeline first checks for existing character_portraits_registry.json. If present, it loads the registry and skips regeneration, optimizing for repeated runs.

Concurrent Generation

When generation is required, the pipeline:

1. Iterates over every CharacterInScene lacking registry entries
2. Launches generate_portraits_for_single_character concurrently for each character
3. Executes sequential generation within each character task: front → side → back

The resulting images are saved to working_dir/character_portraits/<idx>_<identifier>/ before being registered in the global JSON structure (lines 80-108).

Registry Consumption

During frame generation, generate_frames_for_single_camera accesses the registry to supply available_image_path_and_text_pairs for reference-image selection, ensuring characters appear consistent across different camera angles in the final video.

Code Examples

Creating the Registry

pipeline = Script2VideoPipeline(config)
registry = await pipeline.generate_character_portraits(
    characters=extracted_characters,
    character_portraits_registry=None,   # forces fresh generation if no file

    style="studio anime"
)

# `registry` now matches the JSON structure shown above

Generating a Side Portrait After Front Exists

front_path = "/tmp/character_portraits/0_hero/front.png"
side_output = await CharacterPortraitsGenerator(image_gen).generate_side_portrait(
    character=hero_character,
    front_image_path=front_path
)
side_output.save("/tmp/character_portraits/0_hero/side.png")

Reading the Persisted Registry

import json
import os

reg_path = os.path.join(workdir, "character_portraits_registry.json")
with open(reg_path, "r", encoding="utf-8") as f:
    registry = json.load(f)

hero_views = registry["hero"]
print(hero_views["front"]["path"])  # → ".../front.png"

Summary

• ViMax character portrait registry stores front, side, and back views as JSON with file paths and descriptions for each character.
• View generation occurs sequentially: front first, then side and back using the front image as a reference to maintain identity consistency.
• Persistence via character_portraits_registry.json allows reuse across pipeline runs, stored in pipelines/script2video_pipeline.py.
• Error resilience comes from tenacity decorators wrapping all generation methods in agents/character_portraits_generator.py.
• Pipeline integration enables the frame generation step to select appropriate reference images based on camera angles.

Frequently Asked Questions

Where is the ViMax character portrait registry stored?

The registry persists as character_portraits_registry.json in the working directory. The generate_character_portraits method in pipelines/script2video_pipeline.py handles both reading existing registries and writing new ones, making the system restart-friendly.

How does ViMax maintain character consistency across different views?

ViMax maintains consistency through reference image chaining. The generate_side_portrait and generate_back_portrait methods explicitly pass the front view's file path as reference_image_paths to the image generation agent. This technique, implemented in agents/character_portraits_generator.py, ensures hair color, clothing, and facial structure remain identical while only the viewing angle changes.

What happens if image generation fails during portrait creation?

All portrait generation methods are decorated with @retry(stop=stop_after_attempt(3), …) from the tenacity library. If the AI service returns an error or timeout, the system automatically retries up to three times before failing, preventing temporary network issues from breaking the entire video generation pipeline.

How does the pipeline use the registry during frame generation?

During the generate_frames_for_single_camera phase, the pipeline queries the registry to populate available_image_path_and_text_pairs. This allows the system to select the appropriate view (front, side, or back) based on the camera angle described in the shot, ensuring the generated frame references the correct character orientation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}