How Book-to-Skill Detects a Table of Contents (ToC): Multilingual Pattern Matching Explained

Book-to-skill detects a Table of Contents by scanning the first 30 KB of extracted text for multilingual ToC headers using a compiled case-insensitive regex pattern defined in utils.py.

The open-source tool virgiliojr94/book-to-skill extracts structured content from books and technical documents to facilitate skill mapping. A critical first step in this pipeline is determining whether a source document contains a Table of Contents (ToC), which guides subsequent chapter detection and structure analysis. Understanding how this detection works helps users optimize their documents for processing and interpret the tool's metadata outputs correctly.

The ToC Detection Algorithm

Book-to-skill determines whether a document contains a Table of Contents during the structure detection phase performed immediately after raw text extraction. The algorithm relies on pattern matching against a curated, multilingual list of header variations.

Multilingual Header Definitions

In book_to_skill/utils.py (lines 56-64), the detection system uses a tuple named _TOC_HEADERS that enumerates common ToC headings across multiple languages. This comprehensive list covers English, Spanish, Portuguese, Chinese, Japanese, French, German, Italian, and Dutch variants.

_TOC_HEADERS = (
    "table of contents", "contents", "índice", "sumário",   # EN / ES / PT

    "目录", "目錄", "目次",                                 # Chinese / Japanese

    "table des matières",                                 # French

    "inhaltsverzeichnis",                                 # German

    "indice", "sommario",                                 # Italian

    "inhoudsopgave",                                      # Dutch

)

Regex Pattern Compilation

The header list is compiled into a case-insensitive, multiline regular expression stored in _TOC_PATTERN (lines 66-69). This pattern matches any whole line containing one of the specified headers, allowing for optional surrounding whitespace.

_TOC_PATTERN = re.compile(
    r"^\s*(?:" + "|".join(re.escape(h) for h in _TOC_HEADERS) + r")\s*$",
    re.IGNORECASE | re.MULTILINE,
)

Scanning the Document Text

The detect_structure(text: str) -> dict function performs the actual detection. For performance optimization, it scans only the first approximately 30 KB of the extracted text—a trade-off that catches the ToC (typically located at the beginning of documents) while maintaining processing speed and memory efficiency.

# Look for ToC indicators in the first ~30k chars (multilingual; see _TOC_PATTERN)

has_toc = bool(_TOC_PATTERN.search(text[:30000]))

Integration with the Extraction Pipeline

After a file is processed by extract_single_file(), the pipeline calls detect_structure(text) and merges the results into the per-file metadata dictionary. The function returns a structure dictionary containing the has_toc boolean flag alongside other structural metadata.

tokens = estimate_tokens(text)
structure = detect_structure(text)
...
return {
    ...
    **structure,
}

Source: book_to_skill/utils.py (lines 31-49)

User Feedback and Metadata Output

The detection result surfaces to users through both CLI output and structured metadata files. When processing completes, the main entry point prints the ToC status, and if no ToC is found, emits a warning that downstream chapter mapping may rely solely on heading scans.

print(f"   ToC     : {'yes' if consolidated_structure['has_toc'] else 'not detected'}")
if not consolidated_structure["has_toc"]:
    print("   WARN    : No table of contents detected …")

Source: book_to_skill/__main__.py

The generated metadata.json includes the has_toc flag as a top-level boolean value:

{
  "has_toc": true,
  "chapters_detected": 12
}

You can verify detection programmatically using the utility function directly:

from book_to_skill.utils import detect_structure

sample = "Table of Contents\n1. Intro\n2. Body"
result = detect_structure(sample)
print(result["has_toc"])  # Output: True

Or process files in batch to check their ToC status:

from pathlib import Path
from book_to_skill.utils import extract_single_file

for path in Path("books").rglob("*.pdf"):
    info = extract_single_file(path, extraction_mode="text", install_mode="ask")
    print(f"{path.name}: ToC detected? {info['has_toc']}")

Summary

• Multilingual Support: Book-to-skill uses the _TOC_HEADERS tuple in book_to_skill/utils.py covering 10+ languages including English, Spanish, Portuguese, Chinese, and Japanese to identify ToC signals.
• Pattern Matching: A compiled regex _TOC_PATTERN performs case-insensitive, line-anchored matching against document text using re.IGNORECASE | re.MULTILINE flags.
• Performance Optimization: The detect_structure() function scans only the first 30,000 characters (text[:30000]) to balance accuracy and speed.
• Pipeline Integration: Results integrate into the extraction pipeline via extract_single_file() and surface in CLI output and the metadata.json file as the has_toc boolean.

Frequently Asked Questions

What languages does the ToC detection support?

According to the book_to_skill/utils.py source code, the _TOC_HEADERS tuple includes variants for English ("table of contents", "contents"), Spanish ("índice"), Portuguese ("sumário", "índice"), Simplified Chinese ("目录"), Traditional Chinese ("目錄"), Japanese ("目次"), French ("table des matières"), German ("inhaltsverzeichnis"), Italian ("indice", "sommario"), and Dutch ("inhoudsopgave").

Why does book-to-skill only scan the first 30 KB of text?

The 30 KB limit in detect_structure() is a deliberate performance trade-off detailed in the source comments. Since Tables of Contents typically appear at the beginning of books and documents, scanning the first 30,000 characters catches the vast majority of ToCs while keeping the search fast and memory-efficient for large files.

How can I check if my document has a ToC programmatically?

Import detect_structure from book_to_skill.utils and pass your extracted text string to it. The function returns a dictionary with a has_toc key containing a boolean value. For CLI verification, run book-to-skill mybook.pdf and check the output line beginning with ToC :.

What happens if no Table of Contents is detected?

When has_toc evaluates to false, the CLI prints a warning stating that no table of contents was detected. As implemented in the main execution flow, downstream processing may rely only on heading scans for chapter mapping, which can be less precise than ToC-based extraction methods.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}