# Performance Implications of Choosing Different Models in PageIndex: A Technical Analysis

> Explore PageIndex model performance: Analyze tokenization accuracy, API latency, costs & TOC extraction. Understand how LLM choices impact your VectifyAI/PageIndex project.

- Repository: [Vectify AI/PageIndex](https://github.com/vectifyai/pageindex)
- Tags: performance
- Published: 2026-02-16

---

**Choosing different LLM models in PageIndex directly impacts tokenization accuracy, API latency, operational costs, and TOC extraction quality through model-specific encoders and context window constraints.**

PageIndex, an open-source hierarchical table-of-contents (TOC) generator for PDFs from VectifyAI, relies on repeated calls to OpenAI's Chat Completion API to analyze document structure. The specific model you configure—whether `gpt-4o-2024-11-20`, `gpt-4o-mini`, or `gpt-3.5-turbo`—determines how the system tokenizes text, manages context windows, and balances speed against extraction accuracy.

## How Model Selection Affects Tokenization and Pagination

PageIndex uses `tiktoken` to count tokens before sending prompts to the API. In [`pageindex/utils.py`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/utils.py), the `count_tokens` function calls `tiktoken.encoding_for_model(model)` to obtain the correct encoder for your specific model.

```python

# From pageindex/utils.py lines 22-27

def count_tokens(text, model):
    try:
        encoding = tiktoken.encoding_for_model(model)
    except KeyError:
        encoding = tiktoken.get_encoding("cl100k_base")
    return len(encoding.encode(text))

```

Different models use different tokenizers. For example, `gpt-4o` uses the `o200k_base` encoder, while older models use `cl100k_base`. This affects the `max_token_num_each_node` calculation in [`pageindex/page_index.py`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/page_index.py). If you switch models without adjusting this parameter, you risk either under-utilizing available context window or exceeding the model's limits, causing prompt truncation.

## Latency and Throughput Considerations

PageIndex generates TOCs by issuing numerous short prompts for title checking, TOC extraction, and summary generation. The cumulative latency varies significantly based on model choice:

- **gpt-4o-2024-11-20**: 2-4 seconds per call, highest accuracy
- **gpt-4o-mini**: 1-2 seconds per call, balanced performance
- **gpt-3.5-turbo**: <1 second per call, fastest but less reliable

In [`pageindex/utils.py`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/utils.py), all LLM calls (`ChatGPT_API`, `ChatGPT_API_async`, `ChatGPT_API_with_finish_reason`) forward the `model` argument directly to the OpenAI client:

```python

# From pageindex/utils.py lines 29-48

def ChatGPT_API_async(prompt, model, temperature=0.0, ...):
    # ... setup ...

    response = client.chat.completions.create(
        model=model,  # Model passed here

        messages=messages,
        # ...

    )
    return response.choices[0].message.content

```

For processing large PDFs with hundreds of pages, choosing `gpt-4o-mini` can reduce total processing time from minutes to seconds compared to `gpt-4o`, while `gpt-3.5-turbo` offers the fastest throughput for batch operations where perfect accuracy is less critical.

## Cost Analysis and Token Economics

PageIndex processes every page twice: once to count tokens using `count_tokens`, and once to query the model. The total cost scales with both input and output token counts, making model selection a primary cost driver.

The model configuration originates in [`pageindex/config.yaml`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/config.yaml):

```yaml

# From pageindex/config.yaml line 1

model: "gpt-4o-2024-11-20"

```

This value propagates through `page_index_main` in [`pageindex/page_index.py`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/page_index.py) (lines 58-66) to every API call. When you specify a cheaper model like `gpt-3.5-turbo`, you reduce per-token costs by approximately 90% compared to `gpt-4o`, though you may sacrifice extraction accuracy on complex documents with ambiguous headings.

## Accuracy and Context Window Constraints

More capable models better understand ambiguous headings, fuzzy matches, and long prompts. PageIndex uses the `check_title_appearance` function in [`pageindex/page_index.py`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/page_index.py) (lines 13-40) to perform fuzzy matching for title validation:

```python

# Conceptual flow from pageindex/page_index.py

def check_title_appearance(title, page_text, model):
    prompt = f"Does the title '{title}' appear in the following text..."
    # Calls ChatGPT_API_async with the specified model

    return ChatGPT_API_async(prompt, model=model)

```

Weaker models may return false negatives during this check, causing the final TOC to miss sections. Additionally, the default token limit for your chosen model determines the maximum prompt size. PageIndex splits large documents in `page_list_to_group_text` (lines 18-50 of [`pageindex/page_index.py`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/page_index.py)) based on `max_token_num_each_node` (default 20,000). If you switch to a model with a smaller context window (e.g., 8k), you must lower `max_token_num_each_node` or increase the number of groups to prevent prompt truncation.

## Practical Configuration Examples

### Running PageIndex with the Default Model

```python
from pageindex.page_index import page_index

# Uses config.yaml model = "gpt-4o-2024-11-20"

toc = page_index("annual_report.pdf")
print(toc["structure"])

```

### Switching to a Cost-Effective Model

```python
from pageindex.page_index import page_index

# Override for faster, cheaper processing

toc = page_index(
    "simple_document.pdf",
    model="gpt-4o-mini",
    max_token_num_each_node=16000  # Adjust for model context window

)

```

### Optimizing for Speed with GPT-3.5-Turbo

```python
from pageindex.page_index import page_index

# Fastest option for batch processing

toc = page_index(
    "batch_001.pdf",
    model="gpt-3.5-turbo",
    max_token_num_each_node=4000  # Critical: respect 4k context window

)

```

### Measuring Performance Impact

```python
import time
from pageindex.page_index import page_index
from pageindex.utils import count_tokens

pdf_path = "technical_manual.pdf"

# Benchmark GPT-4o

start = time.time()
result_4o = page_index(pdf_path, model="gpt-4o-2024-11-20")
time_4o = time.time() - start

# Benchmark GPT-4o-mini

start = time.time()
result_mini = page_index(pdf_path, model="gpt-4o-mini")
time_mini = time.time() - start

print(f"GPT-4o time: {time_4o:.2f}s")
print(f"GPT-4o-mini time: {time_mini:.2f}s")
print(f"Speed improvement: {(time_4o/time_mini - 1)*100:.0f}%")

```

## Summary

- **Tokenization varies by model**: PageIndex uses `tiktoken.encoding_for_model()` in [`pageindex/utils.py`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/utils.py) to ensure accurate token counts, but different models encode the same text with different token lengths, affecting pagination logic.

- **Latency scales with model capability**: Each LLM call in [`pageindex/utils.py`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/utils.py) forwards your chosen model to the OpenAI client; expect 2-4 seconds per call for `gpt-4o`, 1-2 seconds for `gpt-4o-mini`, and sub-second for `gpt-3.5-turbo`.

- **Cost doubles through token counting**: PageIndex processes each page twice (once for `count_tokens`, once for generation), making the per-token price differential between models significant for large documents.

- **Accuracy depends on model strength**: The `check_title_appearance` function in [`pageindex/page_index.py`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/page_index.py) relies on the model's ability to perform fuzzy matching; weaker models increase false negatives in TOC extraction.

- **Context windows require configuration**: When using models with smaller context windows (e.g., 8k), you must adjust `max_token_num_each_node` in `page_list_to_group_text` (lines 18-50 of [`pageindex/page_index.py`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/page_index.py)) to prevent prompt truncation.

## Frequently Asked Questions

### Does PageIndex support models other than OpenAI GPT?

Currently, PageIndex is designed specifically for OpenAI's Chat Completion API. The `ChatGPT_API` and `ChatGPT_API_async` functions in [`pageindex/utils.py`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/utils.py) directly instantiate the OpenAI client and pass the model string to it. While the architecture could theoretically support other providers through adapter patterns, the current implementation in [`pageindex/utils.py`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/utils.py) (lines 29-48) hardcodes OpenAI-specific client initialization and response handling.

### How do I switch from GPT-4o to GPT-4o-mini in PageIndex?

You can override the default model by passing the `model` parameter directly to the `page_index` function or by modifying [`pageindex/config.yaml`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/config.yaml). For programmatic switching, use `page_index("document.pdf", model="gpt-4o-mini")`. The `page_index_main` function in [`pageindex/page_index.py`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/page_index.py) (lines 58-66) merges your configuration with defaults and propagates the model name to all subsequent API calls in [`utils.py`](https://github.com/VectifyAI/PageIndex/blob/main/utils.py).

### Why does my TOC extraction fail with certain models?

TOC extraction failures typically occur when using models with small context windows (like older GPT-3.5 variants with 4k limits) without adjusting `max_token_num_each_node`. The `page_list_to_group_text` function in [`pageindex/page_index.py`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/page_index.py) (lines 18-50) splits documents based on this parameter; if the value exceeds the model's context window, prompts get truncated, causing the `check_title_appearance` function to miss sections or return malformed JSON. Additionally, weaker models may fail at the fuzzy matching required for title verification.

### What is the optimal max_token_num_each_node for GPT-3.5-Turbo?

For `gpt-3.5-turbo` with its 4,096 token context window, set `max_token_num_each_node` to approximately 2,000-3,000 tokens to leave room for the system prompt and response tokens. In [`pageindex/page_index.py`](https://github.com/VectifyAI/PageIndex/blob/main/pageindex/page_index.py), the `page_list_to_group_text` function uses this parameter to group pages before sending them to the LLM. Setting this too high (e.g., the default 20,000) will cause prompt truncation and failed extractions, while setting it too low increases API call count and cost.