# How book-to-skill Detects Table of Contents: Multilingual Regex Implementation Explained

> Discover how book-to-skill detects tables of contents using a multilingual regex. Learn about its implementation and effectiveness across eight languages plus Chinese.

- Repository: [Virgilio Junior/book-to-skill](https://github.com/virgiliojr94/book-to-skill)
- Tags: internals
- Published: 2026-08-23

---

**Book-to-skill uses a single compiled multilingual regular expression to check the first 30,000 characters of extracted text for common table of contents headings across eight languages plus Chinese.**

The `book-to-skill` library provides automated extraction of structured content from technical books and documentation. A critical step in this pipeline is determining whether a source document contains a table of contents, which influences downstream processing and metadata generation. This detection relies on a carefully designed, language-spanning pattern match rather than heavy document parsing.

## The Detection Pipeline

The table of contents detection in `book-to-skill` follows a streamlined three-stage pipeline: define multilingual headers, compile them into a single regex, and scan the document opening.

### Multilingual Header Vocabulary in [`utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/utils.py)

All header strings are defined as a tuple in **[`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py)** at lines 260–267. The vocabulary covers English, Spanish, Portuguese, French, German, Italian, Dutch, and a dedicated CJK pattern for Chinese variants:

```python

# https://github.com/virgiliojr94/book-to-skill/blob/master/book_to_skill/utils.py#L260-L267

_TOC_HEADERS = (
    "table of contents", "contents", "índice", "sumário",   # EN / ES / PT

    "sumario",                                           # PT (no accent)

    "table des matières",                                 # French

    "inhaltsverzeichnis",                                 # German

    "indice", "sommario",                                 # Italian

    "inhoudsopgave",                                      # Dutch

)
_TOC_CJK_PATTERN = r"目[ \t\u3000]*(?:录|錄|目次)"          # Chinese “目录/目錄/目次”

```

### Compiled Regex Pattern

The headers and CJK pattern are combined into a single **multiline, case-insensitive** regular expression at lines 269–274:

```python

# https://github.com/virgiliojr94/book-to-skill/blob/master/book_to_skill/utils.py#L269-L274

_TOC_PATTERN = re.compile(
    r"^\s*(?:" + "|".join([*(re.escape(h) for h in _TOC_HEADERS), _TOC_CJK_PATTERN]) + r")\s*$",
    re.IGNORECASE | re.MULTILINE,
)

```

The pattern requires:
- The entire line consists only of a recognized header
- Leading and trailing whitespace is ignored
- Case is ignored (`re.IGNORECASE`)
- Multiple lines are checked (`re.MULTILINE`)

### The 30KB Scan in `detect_structure()`

The actual detection occurs in **`detect_structure()`** at lines 64–66. The function searches only the first 30,000 characters for performance and reliability:

```python

# https://github.com/virgiliojr94/book-to-skill/blob/master/book_to_skill/utils.py#L64-L66

has_toc = bool(_TOC_PATTERN.search(text[:30000]))

```

This window size captures the front matter of most technical books while avoiding false positives from body content that might reference "table of contents" in running text.

## Metadata Propagation and Consolidation

The boolean result flows through the extraction pipeline with specific consolidation rules.

### Per-Source Metadata

Each extracted source returns metadata via `detect_structure()` including the `"has_toc"` key. This enables per-file tracking of document structure.

### Multi-Source Book Consolidation

When a book aggregates multiple source files, the consolidated metadata uses `any()` logic at lines 1038–1046:

```python

# https://github.com/virgiliojr94/book-to-skill/blob/master/book_to_skill/utils.py#L1038-L1046

consolidated_structure["has_toc"] = any(src["has_toc"] for src in extracted_sources)

```

This means a book is marked as having a table of contents if **any** source file contains one—appropriate for multi-chapter works where the ToC may appear only in the main file.

## Reuse in Discovery Tax Tool

The same pattern is imported unchanged in **[`tools/discovery_tax.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/discovery_tax.py)** for cost-analysis workflows:

```python

# https://github.com/virgiliojr94/book-to-skill/blob/master/tools/discovery_tax.py#L42-L48

from book_to_skill.utils import (
    _chapter_number as chapter_number,
    _TOC_PATTERN as toc_pattern,
)

```

This ensures consistent detection logic across the codebase without duplication.

## Practical Code Examples

### Basic Detection with `detect_structure()`

```python
from book_to_skill.utils import detect_structure

# English table of contents

txt = "Table of Contents\n1. Introduction\n2. Usage\n"
metadata = detect_structure(txt)
print(metadata["has_toc"])           # → True

# French with surrounding whitespace

txt = "   Table des matières   \nChapitre 1 – Présentation\n"
metadata = detect_structure(txt)
print(metadata["has_toc"])           # → True

# Prose without table of contents

txt = "Once upon a time there was a rabbit..."
metadata = detect_structure(txt)
print(metadata["has_toc"])           # → False

```

### Direct Pattern Usage

For lightweight checks, import `_TOC_PATTERN` directly:

```python
import re
from book_to_skill.utils import _TOC_PATTERN

def contains_toc(text: str) -> bool:
    # Mirror the library's 30KB window behavior

    return bool(_TOC_PATTERN.search(text[:30_000]))

print(contains_toc("Inhaltsverzeichnis\n1 Einführung"))   # True

print(contains_toc("目 录\n第一章")))                      # True (CJK)

```

## Key Implementation Files

| File | Lines | Role in Table of Contents Detection |
|------|-------|-------------------------------------|
| [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py) | 260–274 | Defines header vocabulary, compiles `_TOC_PATTERN` |
| [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py) | 64–66 | Applies pattern in `detect_structure()` |
| [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py) | 1038–1046 | Consolidates multi-source `has_toc` with `any()` |
| [`tools/discovery_tax.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/discovery_tax.py) | 42–48 | Reuses pattern as `toc_pattern` |

## Summary

- **Multilingual coverage**: Eight languages plus Chinese CJK patterns in a single regex
- **Performance constraint**: Only the first 30,000 characters are scanned
- **Strict matching**: Entire line must match; whitespace and case are normalized
- **Consolidation logic**: Multi-source books use `any()` to determine final `has_toc` status
- **Code reuse**: Same pattern exported for downstream tools in [`discovery_tax.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/discovery_tax.py)

## Frequently Asked Questions

### What languages does book-to-skill's table of contents detection support?

English, Spanish, Portuguese, French, German, Italian, Dutch, and Chinese. The Chinese support covers three character variants: 目录, 目錄 (traditional), and 目次—matched via a dedicated regex pattern rather than literal strings.

### Why does the detection only scan the first 30,000 characters?

Technical books typically place the table of contents in front matter. Limiting the scan to 30KB provides **fast rejection** of documents without ToCs while avoiding false positives from later sections that might mention "contents" in conversational text. This threshold is hardcoded in `detect_structure()` at line 64.

### Can the pattern match table of contents headers embedded in sentences?

No. The regex uses `^\s*...pattern...\s*$` with multiline anchors, requiring the header to occupy an entire line. This prevents matches like "See the table of contents for details" from triggering false positives.

### How are multi-file books handled if only one file has a table of contents?

The consolidation logic in [`utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/utils.py) lines 1038–1046 applies `any()` across all sources, so the aggregated book metadata sets `has_toc: true` if any single source reports a match. This design assumes the primary ToC appears once per book, not per chapter.