How book-to-skill Detects Table of Contents: Multilingual Regex Implementation Explained

Book-to-skill uses a single compiled multilingual regular expression to check the first 30,000 characters of extracted text for common table of contents headings across eight languages plus Chinese.

The book-to-skill library provides automated extraction of structured content from technical books and documentation. A critical step in this pipeline is determining whether a source document contains a table of contents, which influences downstream processing and metadata generation. This detection relies on a carefully designed, language-spanning pattern match rather than heavy document parsing.

The Detection Pipeline

The table of contents detection in book-to-skill follows a streamlined three-stage pipeline: define multilingual headers, compile them into a single regex, and scan the document opening.

Multilingual Header Vocabulary in utils.py

All header strings are defined as a tuple in book_to_skill/utils.py at lines 260–267. The vocabulary covers English, Spanish, Portuguese, French, German, Italian, Dutch, and a dedicated CJK pattern for Chinese variants:

# https://github.com/virgiliojr94/book-to-skill/blob/master/book_to_skill/utils.py#L260-L267

_TOC_HEADERS = (
    "table of contents", "contents", "índice", "sumário",   # EN / ES / PT

    "sumario",                                           # PT (no accent)

    "table des matières",                                 # French

    "inhaltsverzeichnis",                                 # German

    "indice", "sommario",                                 # Italian

    "inhoudsopgave",                                      # Dutch

)
_TOC_CJK_PATTERN = r"目[ \t\u3000]*(?:录|錄|目次)"          # Chinese “目录/目錄/目次”

Compiled Regex Pattern

The headers and CJK pattern are combined into a single multiline, case-insensitive regular expression at lines 269–274:

# https://github.com/virgiliojr94/book-to-skill/blob/master/book_to_skill/utils.py#L269-L274

_TOC_PATTERN = re.compile(
    r"^\s*(?:" + "|".join([*(re.escape(h) for h in _TOC_HEADERS), _TOC_CJK_PATTERN]) + r")\s*$",
    re.IGNORECASE | re.MULTILINE,
)

The pattern requires:

• The entire line consists only of a recognized header
• Leading and trailing whitespace is ignored
• Case is ignored (re.IGNORECASE)
• Multiple lines are checked (re.MULTILINE)

The 30KB Scan in detect_structure()

The actual detection occurs in detect_structure() at lines 64–66. The function searches only the first 30,000 characters for performance and reliability:

# https://github.com/virgiliojr94/book-to-skill/blob/master/book_to_skill/utils.py#L64-L66

has_toc = bool(_TOC_PATTERN.search(text[:30000]))

This window size captures the front matter of most technical books while avoiding false positives from body content that might reference "table of contents" in running text.

Metadata Propagation and Consolidation

The boolean result flows through the extraction pipeline with specific consolidation rules.

Per-Source Metadata

Each extracted source returns metadata via detect_structure() including the "has_toc" key. This enables per-file tracking of document structure.

Multi-Source Book Consolidation

When a book aggregates multiple source files, the consolidated metadata uses any() logic at lines 1038–1046:

# https://github.com/virgiliojr94/book-to-skill/blob/master/book_to_skill/utils.py#L1038-L1046

consolidated_structure["has_toc"] = any(src["has_toc"] for src in extracted_sources)

This means a book is marked as having a table of contents if any source file contains one—appropriate for multi-chapter works where the ToC may appear only in the main file.

Reuse in Discovery Tax Tool

The same pattern is imported unchanged in tools/discovery_tax.py for cost-analysis workflows:

# https://github.com/virgiliojr94/book-to-skill/blob/master/tools/discovery_tax.py#L42-L48

from book_to_skill.utils import (
    _chapter_number as chapter_number,
    _TOC_PATTERN as toc_pattern,
)

This ensures consistent detection logic across the codebase without duplication.

Practical Code Examples

Basic Detection with detect_structure()

from book_to_skill.utils import detect_structure

# English table of contents

txt = "Table of Contents\n1. Introduction\n2. Usage\n"
metadata = detect_structure(txt)
print(metadata["has_toc"])           # → True

# French with surrounding whitespace

txt = "   Table des matières   \nChapitre 1 – Présentation\n"
metadata = detect_structure(txt)
print(metadata["has_toc"])           # → True

# Prose without table of contents

txt = "Once upon a time there was a rabbit..."
metadata = detect_structure(txt)
print(metadata["has_toc"])           # → False

Direct Pattern Usage

For lightweight checks, import _TOC_PATTERN directly:

import re
from book_to_skill.utils import _TOC_PATTERN

def contains_toc(text: str) -> bool:
    # Mirror the library's 30KB window behavior

    return bool(_TOC_PATTERN.search(text[:30_000]))

print(contains_toc("Inhaltsverzeichnis\n1 Einführung"))   # True

print(contains_toc("目 录\n第一章")))                      # True (CJK)

Key Implementation Files

File	Lines	Role in Table of Contents Detection
book_to_skill/utils.py	260–274	Defines header vocabulary, compiles _TOC_PATTERN
book_to_skill/utils.py	64–66	Applies pattern in detect_structure()
book_to_skill/utils.py	1038–1046	Consolidates multi-source has_toc with any()
tools/discovery_tax.py	42–48	Reuses pattern as toc_pattern

Summary

• Multilingual coverage: Eight languages plus Chinese CJK patterns in a single regex
• Performance constraint: Only the first 30,000 characters are scanned
• Strict matching: Entire line must match; whitespace and case are normalized
• Consolidation logic: Multi-source books use any() to determine final has_toc status
• Code reuse: Same pattern exported for downstream tools in discovery_tax.py

Frequently Asked Questions

What languages does book-to-skill's table of contents detection support?

English, Spanish, Portuguese, French, German, Italian, Dutch, and Chinese. The Chinese support covers three character variants: 目录, 目錄 (traditional), and 目次—matched via a dedicated regex pattern rather than literal strings.

Why does the detection only scan the first 30,000 characters?

Technical books typically place the table of contents in front matter. Limiting the scan to 30KB provides fast rejection of documents without ToCs while avoiding false positives from later sections that might mention "contents" in conversational text. This threshold is hardcoded in detect_structure() at line 64.

Can the pattern match table of contents headers embedded in sentences?

No. The regex uses ^\s*...pattern...\s*$ with multiline anchors, requiring the header to occupy an entire line. This prevents matches like "See the table of contents for details" from triggering false positives.

How are multi-file books handled if only one file has a table of contents?

The consolidation logic in utils.py lines 1038–1046 applies any() across all sources, so the aggregated book metadata sets has_toc: true if any single source reports a match. This design assumes the primary ToC appears once per book, not per chapter.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}