# Detecting Table of Contents (TOC) in book-to-skill: Multilingual Regex Logic Explained

> Discover the multilingual regex logic book-to-skill uses to detect Tables of Contents (TOC) in book documents. Learn how it scans text for TOC headings efficiently.

- Repository: [Virgilio Junior/book-to-skill](https://github.com/virgiliojr94/book-to-skill)
- Tags: deep-dive
- Published: 2026-08-30

---

**book-to-skill determines whether a source document contains a Table of Contents by scanning the first ~30 KB of extracted plain text for multilingual TOC headings using a compiled regex pattern defined in [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py).**

The open-source repository `virgiliojr94/book-to-skill` provides a robust pipeline for extracting structured knowledge from books and documents. Understanding the logic for detecting Table of Contents (TOC) in book-to-skill is essential for users processing multilingual documents or optimizing extraction workflows. The detection mechanism relies on a lightweight, regex-based approach that balances accuracy with performance across nine different languages.

## The Multilingual TOC Header Dictionary

At the core of the detection system lies **`_TOC_HEADERS`**, a comprehensive tuple defined in [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py) at lines 302-308. This constant enumerates common TOC titles across multiple Latin-script languages to maximize match probability regardless of text extraction quality.

```python
_TOC_HEADERS = (
    "table of contents", "contents", "índice", "sumário",   # EN / ES / PT

    "sumario",                                            # PT (no accent)

    "table des matières",                                 # French

    "inhaltsverzeichnis",                                 # German

    "indice", "sommario",                                 # Italian

    "inhoudsopgave",                                      # Dutch

)

```

The tuple intentionally includes both accented and unaccented variants—such as "sumário" and "sumario"—to accommodate different PDF extraction engines that may strip diacritics. This defensive programming ensures robust detection across Portuguese-language documents regardless of encoding fidelity.

## CJK Pattern Support and Regex Compilation

For Chinese, Japanese, and Korean (CJK) documents, the system employs a dedicated pattern rather than the Latin-based tuple. Lines 309-311 in [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py) define **`_TOC_CJK_PATTERN`**, which matches the traditional and simplified characters for "contents" (目錄, 目録, 目次).

These components merge into **`_TOC_PATTERN`** (lines 310-315), a compiled regular expression that combines the escaped header strings with the CJK pattern. The regex tolerates optional ATX-style markdown heading markers (`#` through `######`) and uses case-insensitive, multiline matching:

```python
_TOC_PATTERN = re.compile(
    r"^\s*(?:#{1,6}\s*)?(?:" + "|".join([*(re.escape(h) for h in _TOC_HEADERS), _TOC_CJK_PATTERN]) + r")\s*$",
    re.IGNORECASE | re.MULTILINE,
)

```

The **`re.MULTILINE`** flag ensures the caret (`^`) and dollar (`$`) anchors match line boundaries rather than the entire string, while **`re.IGNORECASE`** handles capitalization variations in English and German titles.

## Detection Pipeline Implementation

### The 30 KB Text Window

The **`detect_structure()`** function implements a critical performance optimization by limiting TOC searches to the first 30,000 characters of extracted text. According to lines 651-653 in [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py), the function slices `text[:30000]` before applying the pattern search.

```python

# Look for ToC indicators in the first ~30k chars (multilingual; see _TOC_PATTERN)

has_toc = bool(_TOC_PATTERN.search(text[:30000]))

```

This approach assumes that Tables of Contents appear within the opening pages of books, significantly reducing computational overhead on large documents while maintaining high detection accuracy. The 30 KB window typically covers the first 10-15 pages of extracted text, sufficient for most publishing formats.

### Boolean Detection Logic

The function returns a simple boolean **`has_toc`** flag. If `_TOC_PATTERN.search()` finds a match within the initial text window, the flag is set to `True`; otherwise, it remains `False`. This binary classification feeds directly into downstream processing decisions, such as token budgeting and structural parsing.

### Multi-Source Aggregation

When processing books split across multiple PDFs or source files, book-to-skill consolidates per-source results to determine global document structure. Lines 1125-1133 in [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py) use Python's built-in **`any()`** function to aggregate flags across the `extracted_sources` list:

```python

# has_toc is a per-source property, so it has to be combined per source

consolidated_structure["has_toc"] = any(
    src["has_toc"] for src in extracted_sources
)

```

This aggregation logic ensures that fragmented books are flagged as containing a TOC if any constituent file contains recognized headers, preventing false negatives when chapters are distributed across separate files.

## Practical Usage Examples

To programmatically check a text string for TOC presence:

```python
from book_to_skill.utils import detect_structure

# Simple text containing a French TOC heading

txt = "Table des matières\n1. Introduction\n2. Chapter One\n"
info = detect_structure(txt)
print(info["has_toc"])   # → True

# A document without a recognizable TOC heading

txt2 = "Chapter 1\nOnce upon a time..."
info2 = detect_structure(txt2)
print(info2["has_toc"])  # → False

```

For command-line verification during real book processing:

```bash
python -m book_to_skill path/to/book.pdf --mode text

# The CLI internally calls `detect_structure()` and prints:

#    ToC : yes      # if a TOC was detected

```

The same **`_TOC_PATTERN`** is also reused in [`tools/discovery_tax.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/discovery_tax.py) for token-budget calculations, ensuring consistency between detection and cost-estimation workflows.

## Summary

- book-to-skill detects Table of Contents using a multilingual regex pattern stored in `_TOC_PATTERN` that supports English, Spanish, Portuguese, French, German, Italian, Dutch, and CJK languages.
- Detection is restricted to the first 30 KB of text (`text[:30000]`) within the `detect_structure()` function to optimize performance on large documents.
- The system supports both single-file and multi-source book processing, aggregating per-source boolean flags using `any()` logic defined at lines 1125-1133.
- Core implementation resides in [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py) with comprehensive test coverage in [`tests/test_book_to_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tests/test_book_to_skill.py) and multi-source validation in [`tests/test_multi_source_toc.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tests/test_multi_source_toc.py).

## Frequently Asked Questions

### What languages does book-to-skill support for TOC detection?

The repository supports nine language groups through the `_TOC_HEADERS` tuple and `_TOC_CJK_PATTERN` regex: English, Spanish, Portuguese, French, German, Italian, Dutch, and CJK (Chinese, Japanese, Korean). The Portuguese implementation specifically handles both accented ("sumário") and unaccented ("sumario") variants to accommodate different text extraction engines.

### Why does book-to-skill limit TOC detection to the first 30 KB of text?

Limiting the search to `text[:30000]` optimizes processing performance for large documents while leveraging the publishing convention that Tables of Contents consistently appear within the first few pages. This 30 KB window strikes a balance between computational efficiency and detection coverage, as TOCs rarely appear deep in scholarly or commercial texts.

### How does book-to-skill handle TOC detection across multiple files?

When processing books split across multiple PDFs or text files, each source generates its own `has_toc` boolean during extraction. The pipeline consolidates these individual flags using `any()` logic, marking the entire book as having a TOC if any single file contains detected headers. This prevents false negatives when chapters are distributed across separate source files.

### Where is the TOC detection logic tested?

The detection mechanism includes comprehensive test coverage in [`tests/test_book_to_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tests/test_book_to_skill.py), which validates regex matching across all supported languages. Multi-source aggregation logic is specifically tested in [`tests/test_multi_source_toc.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tests/test_multi_source_toc.py), ensuring that `consolidated_structure["has_toc"]` correctly reflects the union of per-source detections.