# How ViMax Extracts Characters from Scripts and Novels: A Complete Technical Guide

> Discover how ViMax extracts characters from scripts and novels using advanced LLM pipelines. Learn about Pydantic parsing and incremental scene-level merging for efficient data extraction.

- Repository: [✨Data Intelligence Lab@HKU✨/ViMax](https://github.com/HKUDS/ViMax)
- Tags: how-to-guide
- Published: 2026-05-20

---

**ViMax uses two distinct LLM-powered pipelines to extract characters from scripts and novels—direct Pydantic parsing for screenplays via `CharacterExtractor` and incremental scene-level merging via `GlobalInformationPlanner` for novels.**

ViMax is an open-source visual storytelling framework developed by HKUDS that transforms text into cinematic storyboards. To extract characters from scripts and novels reliably, the codebase implements specialized agents in the `agents/` directory that leverage LangChain chat models, structured prompting, and Pydantic output parsing for type-safe character metadata extraction.

## Script Character Extraction

For movie scripts, ViMax employs a single-pass extraction strategy that parses the entire screenplay in one LLM invocation.

### The CharacterExtractor Agent

The `CharacterExtractor` class in [`agents/character_extractor.py`](https://github.com/HKUDS/ViMax/blob/main/agents/character_extractor.py) (lines 15-55) handles script processing through a structured prompt engineering approach:

1. **System prompt** – Defines the extractor’s role, output format requirements, and distinguishes between static features (physical attributes) and dynamic features (emotional states, visibility).
2. **Human prompt** – Wraps the raw script content within `<SCRIPT> … </SCRIPT>` XML tags to delimit the input context.
3. **Pydantic validation** – Uses `PydanticOutputParser` expecting an `ExtractCharactersResponse` containing a list of `CharacterInScene` objects (defined at lines 57-61).

The extracted characters are persisted to [`working_dir/characters.json`](https://github.com/HKUDS/ViMax/blob/main/working_dir/characters.json) for downstream pipeline stages, as implemented in [`pipelines/script2video_pipeline.py`](https://github.com/HKUDS/ViMax/blob/main/pipelines/script2video_pipeline.py) (lines 25-40) where `self.character_extractor.extract_characters(script)` is invoked.

```python
from agents.character_extractor import CharacterExtractor
from langchain.chat_models import init_chat_model

chat = init_chat_model(model="gpt-4o-mini")
extractor = CharacterExtractor(chat_model=chat)

script = """
INT. CAFE – DAY
A young woman sips coffee, eyes distant...
"""
characters = await extractor.extract_characters(script)
print(characters)   # → List[CharacterInScene] with static/dynamic features

```

## Novel Character Extraction

Novel processing requires a more complex, multi-stage approach because source material lacks explicit scene boundaries and dialogue markers.

### Multi-Stage Architecture

The `GlobalInformationPlanner` in [`agents/global_information_planner.py`](https://github.com/HKUDS/ViMax/blob/main/agents/global_information_planner.py) orchestrates character discovery through four sequential phases:

1. **Novel compression** – `NovelCompressor` splits and compresses large texts to fit context windows.
2. **Event extraction** – Identifies narrative events from compressed content.
3. **Scene generation** – `SceneExtractor` converts events into screenplay-formatted scenes.
4. **Hierarchical merging** – Combines character references across scenes and events into a canonical novel-wide list.

### Merging Characters Across Scenes and Events

For each narrative event, `merge_characters_across_scenes_in_event` (lines 58-96) resolves character identity across fragmented scene references. The method serializes scenes using `<SCENE_N_START>` … `</SCENE_N_END>` tags and prompts the LLM to identify which character mentions refer to the same entity, resolving ambiguous naming and evolving descriptions.

### Global Novel-Level Consolidation

The `merge_characters_to_existing_characters_in_novel` method (lines 20-72) performs the final integration step. It constructs structured prompts containing `<EXISTING_CHARACTERS>` and `<EVENT_CHARACTERS>` sections, enabling the LLM to update the global character catalogue with new entries or merge features when the same character appears in multiple events. This logic is invoked iteratively within [`pipelines/novel2movie_pipeline.py`](https://github.com/HKUDS/ViMax/blob/main/pipelines/novel2movie_pipeline.py) (lines 96-124).

```python
from agents.global_information_planner import GlobalInformationPlanner
from langchain.chat_models import init_chat_model

chat = init_chat_model(model="gpt-4o-mini")
planner = GlobalInformationPlanner(
    api_key="YOUR_KEY",
    base_url="https://api.openai.com/v1",
    chat_model="gpt-4o-mini",
)

# Merge characters within a single event from multiple scenes

merged_event_chars = await planner.merge_characters_across_scenes_in_event(
    event_idx=0,
    scenes=scenes,
)

# Integrate event-level characters into the novel-wide roster

novel_chars = await planner.merge_characters_to_existing_characters_in_novel(
    event_idx=0,
    existing_characters_in_novel=novel_chars,
    characters_in_event=merged_event_chars,
)

print(novel_chars)   # → List[CharacterInNovel] representing the whole novel

```

## Key Differences: Script vs. Novel Processing

ViMax maintains separate extraction paths because the input characteristics differ fundamentally:

- **Script extraction** is *immediate*. Screenplays already delineate dialogue headers (e.g., `INT. CAFE – DAY`) and character names, allowing `CharacterExtractor` to map mentions directly to `CharacterInScene` objects in a single LLM pass.
- **Novel extraction** is *discover*. Prose lacks explicit scene boundaries, requiring `GlobalInformationPlanner` to first identify events, generate scenes, and then merge fragmented character references across the narrative timeline to maintain consistency.

Both implementations rely on **LangChain** chat models and **Pydantic** for type-safe output validation, but the novel pipeline adds layers of indirection to handle the ambiguity and scale of book-length content.

## Summary

- **Script extraction** uses `CharacterExtractor` ([`agents/character_extractor.py`](https://github.com/HKUDS/ViMax/blob/main/agents/character_extractor.py)) with `<SCRIPT>` XML tags and direct Pydantic parsing into `CharacterInScene` objects.
- **Novel extraction** relies on `GlobalInformationPlanner` ([`agents/global_information_planner.py`](https://github.com/HKUDS/ViMax/blob/main/agents/global_information_planner.py)) to merge characters hierarchically across scenes and events using structured tags like `<SCENE_N_START>` and `<EXISTING_CHARACTERS>`.
- **Pipeline integration** occurs in [`script2video_pipeline.py`](https://github.com/HKUDS/ViMax/blob/main/script2video_pipeline.py) and [`novel2movie_pipeline.py`](https://github.com/HKUDS/ViMax/blob/main/novel2movie_pipeline.py), with results cached for downstream video generation stages.
- **Type safety** is enforced through `PydanticOutputParser` and response models including `ExtractCharactersResponse` and `CharacterInNovel`.

## Frequently Asked Questions

### What is the difference between CharacterInScene and CharacterInNovel?

`CharacterInScene` (defined in [`agents/character_extractor.py`](https://github.com/HKUDS/ViMax/blob/main/agents/character_extractor.py), lines 57-61) represents character metadata extracted from a specific screenplay segment, capturing immediate context like visibility and emotional state. `CharacterInNovel` represents the canonical character entity maintained across the entire book, containing accumulated attributes from multiple events and scenes through the merging process in `GlobalInformationPlanner`.

### How does ViMax handle character consistency across multiple novel chapters?

ViMax maintains consistency through incremental merging. The `merge_characters_to_existing_characters_in_novel` method compares new event characters against the existing novel-wide roster using `<EXISTING_CHARACTERS>` and `<EVENT_CHARACTERS>` prompt sections. The LLM determines whether to create new entries or update existing character profiles when encountering alternate names, physical description changes, or new narrative roles across different chapters.

### Why does ViMax use PydanticOutputParser instead of raw JSON?

According to the HKUDS/ViMax source code, `PydanticOutputParser` guarantees type-safe deserialization of LLM outputs into structured Python objects like `ExtractCharactersResponse`. This prevents runtime errors from malformed JSON, enforces schema validation for required fields (e.g., character names, static features), and enables IDE autocomplete and type checking throughout the pipeline in [`script2video_pipeline.py`](https://github.com/HKUDS/ViMax/blob/main/script2video_pipeline.py) and [`novel2movie_pipeline.py`](https://github.com/HKUDS/ViMax/blob/main/novel2movie_pipeline.py).

### Can I use custom LLM models with ViMax's character extraction?

Yes. Both `CharacterExtractor` and `GlobalInformationPlanner` accept any LangChain-compatible chat model through the `chat_model` parameter or `init_chat_model()` configuration. You can substitute `gpt-4o-mini` with other OpenAI models, local LLMs via Ollama, or alternative providers by adjusting the `model` string and API configuration in the constructor calls.