# ViMax Pydantic Data Interfaces for Character, Scene, and Shot Descriptions

> Discover ViMax Pydantic data interfaces for character, scene, and shot descriptions. Ensure validated data flow between AI agents with strict, strongly-typed schemas.

- Repository: [✨Data Intelligence Lab@HKU✨/ViMax](https://github.com/HKUDS/ViMax)
- Tags: api-reference
- Published: 2026-05-20

---

**ViMax utilizes strict Pydantic `BaseModel` classes located in the `main/interfaces` package to define strongly-typed schemas for narrative elements, ensuring validated data flows between AI agents without fragile dictionary-based contracts.**

The HKUDS/ViMax repository implements a comprehensive type system for video generation pipelines using Pydantic data interfaces. These interfaces model characters, scenes, and shots as validated Python objects, enabling reliable communication between specialized agents like scene extractors, storyboard artists, and camera image generators.

## Character Interfaces in [`main/interfaces/character.py`](https://github.com/HKUDS/ViMax/blob/main/main/interfaces/character.py)

The character module defines three hierarchical Pydantic models that represent characters at different granularities of the narrative structure.

### CharacterInScene

The `CharacterInScene` class captures a single character's appearance within a specific scene. According to the source code in [`main/interfaces/character.py`](https://github.com/HKUDS/ViMax/blob/main/main/interfaces/character.py), this model enforces strict typing on five primary fields:

- `idx`: Integer index for positional reference
- `identifier_in_scene`: String name or tag used in scripts (e.g., "Alice" or "Bob the Builder")
- `is_visible`: Boolean flag indicating whether the character appears on screen
- `static_features`: String describing immutable characteristics (physical appearance, clothing baseline)
- `dynamic_features`: String describing scene-specific variations (temporary accessories, posture changes)

### CharacterInEvent and CharacterInNovel

For higher-level narrative aggregation, ViMax provides `CharacterInEvent` and `CharacterInNovel` models. These track characters across multiple scenes or the entire novel:

- **CharacterInEvent**: Links characters to events via `index`, `identifier_in_event`, `active_scenes` (List of scene indices), and `static_features`
- **CharacterInNovel**: Provides the broadest scope with `index`, `identifier_in_novel`, `active_events` (List of event indices), and `static_features`

## Scene Interface in [`main/interfaces/scene.py`](https://github.com/HKUDS/ViMax/blob/main/main/interfaces/scene.py)

The `Scene` model acts as a container aggregating environment data, character lists, and script text. Defined in [`main/interfaces/scene.py`](https://github.com/HKUDS/ViMax/blob/main/main/interfaces/scene.py), it consists of:

- `idx`: Scene identifier (int)
- `is_last`: Boolean indicating if this is the final scene
- `environment`: Instance of `EnvironmentInScene` (location, time_of_day, weather)
- `characters`: List of `CharacterInScene` objects
- `script`: Raw text string containing dialogue and action descriptions

This design allows the `scene_extractor` agent to output validated objects that downstream agents can consume without manual parsing.

## Shot Description Interfaces in [`main/interfaces/shot_description.py`](https://github.com/HKUDS/ViMax/blob/main/main/interfaces/shot_description.py)

Shot-level data moves through two distinct Pydantic interfaces representing different pipeline stages.

### ShotBriefDescription

Used by planners during initial storyboarding, `ShotBriefDescription` provides a lightweight placeholder with:

- `idx`: Shot index (int)
- `is_last`: Boolean flag for sequence termination
- `cam_idx`: Camera identifier (int)
- `visual_desc`: String describing the visual composition
- `audio_desc`: String describing ambient sound or music

### ShotDescription

The full `ShotDescription` model expands the brief into production-ready specifications with thirteen strictly-typed fields:

- **Core identifiers**: `idx`, `is_last`, `cam_idx`
- **Variation metadata**: `variation_type` (Literal values like "large", "medium", "small"), `variation_reason` (explanatory string)
- **Frame descriptions**: `ff_desc` (first frame), `lf_desc` (last frame)
- **Character indexing**: `ff_vis_char_idxs` and `lf_vis_char_idxs` (List[int] referencing `CharacterInScene.idx` values)
- **Motion and audio**: `motion_desc` (action details), `audio_desc` (sound design)

The character index lists (`ff_vis_char_idxs`, `lf_vis_char_idxs`) maintain referential integrity to the parent scene's `CharacterInScene` objects via their `idx` fields.

## Working with ViMax Pydantic Interfaces

Instantiating these models provides runtime validation and IDE autocomplete support. Below are practical implementations demonstrating the hierarchical relationships between characters, scenes, and shots.

### Creating Characters for a Scene

```python
from ViMax.main.interfaces.character import CharacterInScene
from ViMax.main.interfaces.scene import Scene
from ViMax.main.interfaces.environment import EnvironmentInScene

alice = CharacterInScene(
    idx=0,
    identifier_in_scene="Alice",
    is_visible=True,
    static_features="Alice has long blonde hair and blue eyes, and is of slender build.",
    dynamic_features="Wearing a red scarf and a black leather jacket",
)

bob = CharacterInScene(
    idx=1,
    identifier_in_scene="Bob the Builder",
    is_visible=False,
    static_features="Bob the Builder is a middle‑aged man with a sturdy build.",
    dynamic_features="",
)

scene = Scene(
    idx=0,
    is_last=False,
    environment=EnvironmentInScene(
        location="Supermarket",
        time_of_day="Afternoon",
        weather="Indoor",
    ),
    characters=[alice, bob],
    script="""
<Alice> looks around the aisle, searching for the missing ingredient.
<Bob the Builder> (off‑screen) mutters: "I think I saw it near the dairy section."
""",
)

```

### Generating Shot Descriptions

Planners first create brief descriptions before expanding to full specifications:

```python
from ViMax.main.interfaces.shot_description import ShotBriefDescription, ShotDescription

# Initial brief from planner

brief = ShotBriefDescription(
    idx=0,
    is_last=False,
    cam_idx=0,
    visual_desc="An over‑the‑shoulder shot behind <Alice>, showing her hand reaching for a can.",
    audio_desc="[Sound Effect] Soft hum of the refrigerator.",
)

# Expanded production description

full = ShotDescription(
    idx=0,
    is_last=False,
    cam_idx=0,
    visual_desc="A close‑up of <Alice>'s hand as she grabs a can of tomatoes.",
    variation_type="small",
    variation_reason="Only the hand changes between frames.",
    ff_desc="Close‑up of an empty shelf.",
    ff_vis_char_idxs=[],
    lf_desc="Close‑up of the can in Alice's hand.",
    lf_vis_char_idxs=[0],  # References alice.idx

    motion_desc="Alice's fingers close around the can.",
    audio_desc="[Sound Effect] Shelf creak followed by a soft click.",
)

```

## Summary

- **ViMax Pydantic data interfaces** reside in [`main/interfaces/character.py`](https://github.com/HKUDS/ViMax/blob/main/main/interfaces/character.py), [`main/interfaces/scene.py`](https://github.com/HKUDS/ViMax/blob/main/main/interfaces/scene.py), and [`main/interfaces/shot_description.py`](https://github.com/HKUDS/ViMax/blob/main/main/interfaces/shot_description.py)
- **Character models** exist at three scopes: `CharacterInScene` (single scene), `CharacterInEvent` (event level), and `CharacterInNovel` (global)
- **Scene objects** aggregate environments, character lists, and scripts using strict `List` and `BaseModel` typing
- **Shot descriptions** progress from `ShotBriefDescription` (planning) to `ShotDescription` (production), with character visibility tracked via integer index references
- All interfaces use Pydantic validation to ensure type safety across agent boundaries

## Frequently Asked Questions

### What fields are required for CharacterInScene validation?

According to [`main/interfaces/character.py`](https://github.com/HKUDS/ViMax/blob/main/main/interfaces/character.py), `CharacterInScene` requires `idx` (int), `identifier_in_scene` (str), `is_visible` (bool), `static_features` (str), and `dynamic_features` (str). The model validates that these fields conform to their specified types at instantiation time.

### How does ShotDescription reference characters from the parent scene?

`ShotDescription` uses integer lists `ff_vis_char_idxs` and `lf_vis_char_idxs` to store indices corresponding to the `idx` field of `CharacterInScene` objects in the parent scene. This maintains lightweight references without duplicating character data.

### Why does ViMax use Pydantic instead of dictionaries for data interfaces?

The ViMax codebase uses Pydantic `BaseModel` classes to provide runtime type validation, IDE autocomplete support, and self-documenting schemas. This prevents runtime errors from missing keys or type mismatches when passing data between agents like `scene_extractor` and `storyboard_artist`.

### Where are the environment models defined in ViMax?

The `EnvironmentInScene` model referenced by `Scene.characters` is defined in [`main/interfaces/environment.py`](https://github.com/HKUDS/ViMax/blob/main/main/interfaces/environment.py) (companion file to the character and scene modules), providing typed fields for `location`, `time_of_day`, and `weather`.