PageIndex Retry Mechanism for Fixing Incorrect TOC Entries: Implementation Guide

PageIndex implements a bounded exponential retry loop via fix_incorrect_toc_with_retries in pageindex/page_index.py that attempts up to 3 iterations to correct inaccurate table-of-contents entries before returning the best-effort result.

The VectifyAI/PageIndex repository provides intelligent document processing capabilities that validate and correct table-of-contents (TOC) accuracy through automated verification. When the initial extraction produces entries with incorrect page numbers, the retry mechanism used by PageIndex for fixing incorrect TOC entries activates to iteratively refine results using a bounded loop with configurable attempt limits.

How the Retry Mechanism Works in PageIndex

Triggering the Retry Loop

The retry process initiates within the meta_processor function (around line 771 in pageindex/page_index.py) after initial TOC verification. When the accuracy score exceeds 0.6 but incorrect entries persist, the system invokes the retry helper:

toc_with_page_number, incorrect_results = await fix_incorrect_toc_with_retries(
    toc_with_page_number,
    page_list,
    incorrect_results,
    start_index=start_index,
    max_attempts=3,
    model=opt.model,
    logger=logger
)

This conditional trigger ensures that retries only occur when the document shows sufficient initial accuracy to warrant correction attempts, avoiding wasted computation on severely malformed inputs.

The Exponential Retry Implementation

The core retry logic resides in fix_incorrect_toc_with_retries (lines 870‑884 in pageindex/page_index.py). This asynchronous function implements a while-loop that continues processing until either all entries are corrected or the attempt limit is reached:

async def fix_incorrect_toc_with_retries(toc, pages, wrong, start_index=1,
                                         max_attempts=3, model=None, logger=None):
    attempt = 0
    while wrong:
        toc, wrong = await fix_incorrect_toc(toc, pages, wrong,
                                             start_index, model, logger)
        attempt += 1
        if attempt >= max_attempts:
            logger.info("Maximum fix attempts reached")
            break
    return toc, wrong

Each iteration calls fix_incorrect_toc (lines 752‑866), which attempts to locate the correct physical page for every flagged entry using the specified LLM model and document content analysis.

Termination Conditions

The retry mechanism terminates under two specific conditions:

• Success state: When current_incorrect becomes empty, indicating all previously flagged TOC entries have been resolved.
• Attempt exhaustion: When the counter reaches max_attempts (default 3), triggering a log entry and returning the best-effort TOC available.

This bounded approach prevents infinite loops on documents with inherently ambiguous page structures while maximizing correction opportunities for fixable errors.

Configuration and Customization

You can customize the retry behavior by adjusting the max_attempts parameter when calling the function directly:

# Custom retry configuration for complex documents

toc, remaining_errors = await fix_incorrect_toc_with_retries(
    initial_toc,
    document_pages,
    incorrect_entries,
    start_index=1,
    max_attempts=5,  # Increase for documents with complex layouts

    model="gpt-4o-2024-11-20",
    logger=custom_logger
)

The model parameter accepts any string identifier compatible with your LLM backend, allowing you to switch between different model versions or providers based on accuracy requirements and latency constraints.

Summary

• PageIndex implements a bounded retry mechanism in fix_incorrect_toc_with_retries to correct inaccurate TOC entries through iterative refinement.
• The retry loop triggers when initial verification scores exceed 0.6 but incorrect entries persist, as determined by meta_processor.
• Each retry attempt invokes fix_incorrect_toc to re-analyze page content and relocate entries, with a default maximum of 3 attempts.
• The mechanism terminates upon successful correction of all entries or exhaustion of the configured attempt limit, ensuring deterministic execution without infinite loops.

Frequently Asked Questions

What triggers the retry mechanism in PageIndex?

The retry mechanism activates within the meta_processor function when the initial TOC verification produces an accuracy score greater than 0.6 but still contains incorrect entries. This threshold ensures that retries only occur on documents with sufficient structural integrity to benefit from correction attempts, avoiding wasted computation on severely malformed inputs.

How many retry attempts does PageIndex perform by default?

By default, PageIndex performs 3 retry attempts as specified by the max_attempts=3 parameter in the fix_incorrect_toc_with_retries function call within meta_processor. This default provides a balance between correction opportunity and processing efficiency for typical document layouts.

Can I customize the maximum number of retry attempts?

Yes, you can customize the retry limit by passing a different value to the max_attempts parameter when calling fix_incorrect_toc_with_retries directly. For complex documents with ambiguous page layouts, increasing this value to 5 or higher may improve final TOC accuracy, though it will increase processing time and LLM API costs proportionally.

What happens if the TOC entries cannot be fixed after all retries?

If entries remain incorrect after exhausting all retry attempts, the function returns the best-effort TOC along with the remaining incorrect entries. The system logs "Maximum fix attempts reached" via the provided logger, allowing downstream processes to handle unresolved entries appropriately—either by flagging them for manual review or proceeding with the partially corrected TOC.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}