How Book-to-Skill Detects Chapters in Documents: Multi-Language Pattern Matching Explained

Book-to-Skill detects chapters by scanning extracted text for language-specific heading patterns using regex matchers in book_to_skill/utils.py, supporting Arabic numerals, Roman numerals, Chinese, Thai, and Korean scripts.

The open-source repository virgiliojr94/book-to-skill provides a robust Python utility for analyzing book structures. Its chapter detection engine handles diverse international publishing formats by recognizing native numeral systems and heading conventions across multiple writing scripts.

The Chapter Detection Pipeline

The detection workflow centers on the _chapter_number() helper function in book_to_skill/utils.py. This function implements a cascading regex strategy to identify chapter boundaries in raw text extracted from PDFs, EPUBs, or DOCX files.

The process follows five distinct stages:

1. Line-level scan – Each line of extracted text is fed individually to _chapter_number().
2. Pattern matching – The function tests regex patterns in sequence: explicit Latin headings, Roman numerals, Chinese, Thai, and Korean.
3. Validation – Matched lines are verified against _HEADING_TAIL to ensure they represent actual headings rather than prose references (e.g., filtering out "Chapter 6 explores...").
4. Number extraction – Captured numerals convert to integers via script-specific helpers (_roman_to_int, _cn_numeral_to_int, Thai digit translation, etc.).
5. Deduplication – The detect_structure() function (lines 92‑99) aggregates distinct chapter numbers, preventing duplicate counts when a Table of Contents entry mirrors a body heading.

Multi-Language Regex Patterns

Book-to-Skill defines compiled regex patterns for each supported language family. These patterns recognize both standalone headings and Markdown-style prefixes.

Latin-Based Languages and Roman Numerals

For English, Spanish, Portuguese, French, German, Italian, and Dutch, the _EXPLICIT_CHAPTER regex (lines 54‑62) captures headings using Arabic or Roman numerals. The pattern handles variations like "Chapter 5", "Capítulo 5: …", and "chapitre 5".

Roman numeral support extends to standalone headings and Markdown formats through _ROMAN_HEAD and _LC_MD_ROMAN (lines 70‑77), recognizing patterns like I: Loomings or ## i. introduction.

Chinese Chapter Headings

Chinese text detection uses _CN_CHAPTER and _MD_CN_HEADING (lines 78‑99) to identify traditional chapter markers. These patterns match full-width digits and ordinal expressions such as 第 3 章 …, ## 一 · 缘起, and ## 第一讲.

Thai Numeral Support

The _TH_CHAPTER regex (lines 100‑108) handles Thai script using a digit-mapping system for Thai numerals. It recognizes structures like บทที่ 3 and ## บทที่ ๑, converting traditional Thai digits to standard integers for sequencing.

Korean Chapter Markers

Korean documents are parsed using _KO_CHAPTER (lines 110‑124), which matches patterns like 제1장 총칙 and ## 제4장 근로시간과 휴식. The regex captures the numeric portion following the "제" prefix and preceding the "장" chapter suffix.

Validation and False-Positive Guards

To prevent misidentification of inline references or list items as chapter headings, the system implements robust validation logic. The _HEADING_TAIL regex verifies that the remainder of a matched line contains end-of-line punctuation or capitalized titles.

Additional safeguards include:

• Length limits – Skipping lines exceeding 80 characters to avoid matching long prose sentences.
• Heading-tail verification – Ensuring the structure matches publication conventions rather than casual mentions.
• Distinct-number aggregation – detect_structure() filters duplicate chapter numbers that might appear in both a table of contents and chapter bodies.

Fallback Structural Detection

When documents lack numeric chapter headings (for example, books using "Chapter One" or pure Markdown with semantic headers), book-to-skill activates _structural_chapter_count. This fallback mechanism counts Markdown or AsciiDoc heading levels (#, ##, etc.) to estimate chapter boundaries, ensuring the pipeline remains functional for non-standard publications.

Integration with the CLI Pipeline

The command-line interface in book_to_skill/cli.py orchestrates the extraction through the extract_single_file function. After parsers extract raw text from source documents, the CLI calls detect_structure() to populate the metadata.json output.

The resulting metadata dictionary includes:

• chapters_detected: Integer count of unique chapters identified.
• has_toc: Boolean indicating whether a Table of Contents structure was found.

Code Examples

You can interact with the chapter detection logic directly through the Python API:

from book_to_skill.utils import _chapter_number, detect_structure

sample = """
Chapter 1: Introduction
第 2 章 基础概念

## 三 · 高级话题

บทที่ 4 การประยุกต์ใช้
제5장 결론
"""

# Show per-line detection

for line in sample.splitlines():
    print(line, "→", _chapter_number(line))

# Detect overall structure

info = detect_structure(sample)
print("\nDetected chapters:", info["chapters_detected"])
print("Table of contents found?:", info["has_toc"])

Output:

Chapter 1: Introduction → 1
第 2 章 基础概念 → 2

## 三 · 高级话题 → 3

บทที่ 4 การประยุกต์ใช้ → 4
제5장 결론 → 5

Detected chapters: 5
Table of contents found?: False

For full pipeline execution via the command line:

# Extract and analyze a full ebook

python -m book_to_skill extract.py my_ebook.epub

# Inspect generated metadata

cat metadata.json | jq '.chapters_detected, .has_toc'

Summary

• Primary detection occurs in book_to_skill/utils.py via the _chapter_number() helper and detect_structure() function.
• Multi-language support includes dedicated regex patterns for Latin/Roman, Chinese, Thai, and Korean scripts with appropriate numeral conversion helpers.
• Validation layers prevent false positives through length checks, heading-tail verification, and distinct-number deduplication.
• Fallback mechanism _structural_chapter_count handles non-numeric headings in Markdown or AsciiDoc formats.
• CLI integration exposes detection results through metadata.json after processing via extract_single_file.

Frequently Asked Questions

What file contains the core chapter detection logic?

The core logic resides in book_to_skill/utils.py. This file defines the _chapter_number() helper, all language-specific regex patterns (e.g., _EXPLICIT_CHAPTER, _CN_CHAPTER), numeral conversion utilities, and the detect_structure() orchestration function.

How does book-to-skill handle Chinese chapter numbers?

Chinese detection uses the _CN_CHAPTER and _MD_CN_HEADING patterns (lines 78‑99) to match headings containing traditional ordinal expressions and full-width digits. The _cn_numeral_to_int() helper converts captured Chinese numerals to integers, supporting formats like 第 3 章 and ## 第一讲.

What happens if a document has no numeric chapter headings?

When numeric headings are absent, the system falls back to _structural_chapter_count, which analyzes Markdown or AsciiDoc heading hierarchies (#, ##, etc.) to estimate chapter boundaries. This ensures the chapters_detected field in metadata.json remains populated even for books using written-out titles like "Chapter One".

Can I use the chapter detector independently of the full CLI pipeline?

Yes. Import _chapter_number() and detect_structure() directly from book_to_skill.utils to analyze raw text strings without invoking the file extraction layer. This approach is demonstrated in tools/discovery_tax.py, which reuses the detection logic for downstream taxonomy estimation tasks.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}