How book-to-skill Detects Document Structure and Chapter Headings: A Code-Level Guide

The book-to-skill library detects document structure and chapter headings using a two-phase strategy implemented in book_to_skill/utils.py: first scanning for explicit multilingual chapter patterns (Arabic numerals, Roman numerals, Chinese "第N章", Korean "제N장", etc.), then falling back to structural Markdown/AsciiDoc heading analysis when explicit patterns are absent.

The book-to-skill repository provides intelligent document parsing that automatically identifies chapter boundaries across dozens of languages. Understanding how it detects document structure and chapter headings reveals a sophisticated pattern-matching system that balances linguistic awareness with structural analysis. This guide examines the implementation details in the source code, showing how the library distinguishes genuine chapter headings from false positives like code examples or decorative text.

The Two-Phase Detection Strategy in detect_structure

The entry point for all structure analysis is the detect_structure() function in book_to_skill/utils.py. This function implements a cascading approach that prioritizes explicit chapter declarations before attempting structural inference.

Phase 1: Multilingual Chapter Number Extraction

The library first attempts to find explicit chapter indicators using the _chapter_number() function (lines 44-55). This helper relies on _match_chapter_number() (lines 17-25) to test lines against a comprehensive suite of regular expressions compiled at module import time (lines 103-135).

As detect_structure iterates over the entire text (lines 84-90), it builds a set of all numeric chapter values found. The size of this set (numeric_count) determines if the document contains explicit chapter declarations. If at least two distinct chapter numbers are found, the function classifies the detection method as "numeric".

Phase 2: Structural Heading Fallback

If no explicit chapter numbers are detected, detect_structure invokes _structural_chapter_count() (lines 83-61) to analyze document markup. This function identifies potential chapters by examining Markdown ATX headings (# Title), Setext-style underlines (===), and AsciiDoc conventions.

The structural analyzer counts headings at the shallowest depth that contains at least two titles, effectively identifying the top-level sectioning scheme while ignoring deeper nested headings.

Regex Patterns for Global Chapter Recognition

The _chapter_number() function recognizes chapter declarations across multiple writing systems through case-insensitive regex patterns compiled during module initialization:

• Latin Scripts: Matches "Chapter", "Chapitre", "Kapitel", "Capítulo", "Capitolo", "Hoofdstuk", and abbreviated forms like "Ch.", capturing both Arabic numerals (\d{1,2}) and Roman numerals ([IVXLCDMivxlcdm]{1,7}).

• Chinese Numerals: Detects the pattern "第 N 章" using ^\s*第\s*([0-9０-９〇零一二两三四五六七八九十百千]+)\s*[章回卷节篇讲], handling both full-width digits and traditional number characters like 零, 一, 二.

• Thai: Recognizes "บทที่" (chapter), "ตอนที่" (episode), and "ภาคที่" (part) followed by Thai digits ([0-9๐-๙]+).

• Hindi (Devanagari): Identifies "अध्याय" (chapter) followed by Devanagari numerals ([0-9०-९]+).

• Korean: Matches the "제 N 장" pattern with ^\s*(?:#{1,6}\s+)?제\s*([0-9]+)\s*[장편절관], handling legislative-style documents.

• Persian: Implements word-ordinal mapping via _FA_ORDINALS to detect patterns like "فصل ۲" or "فصل بیست و یکم".

• Roman Numerals: Supports both standalone uppercase numerals (II. The Carpet-Bag) and Markdown-prefixed lowercase numerals (## i. introduction).

All patterns ignore Markdown heading prefixes (#{1,6}\s+) to ensure compatibility with formatted documents.

Guarding Against False Positives

The library implements several guards in detect_structure to prevent misidentification of non-chapter content:

• Code Fence Exclusion: The _closed_fence_line_numbers() function (lines 20-33) identifies line ranges inside properly closed Markdown code blocks. Headings appearing within these ranges are excluded from structural analysis.

• Digit-Only Title Filtering: When structural detection encounters titles beginning with digits, detect_structure applies the _numbered_titles_are_structural heuristic to distinguish chapter numbers from numbered lists or figure captions.

• Punctuation Validation: Both ATX and Setext detection branches reject titles containing no word characters (re.search(r"\w", title)), preventing decorative separators like === or --- from being classified as chapters.

Table of Contents Detection

Beyond chapter identification, the library detects Tables of Contents using _TOC_PATTERN (lines 87-103). This regex scans the first approximately 30,000 characters for multilingual ToC headings including "Table of Contents", "Índice", the Chinese pattern "目 录", and equivalents.

The boolean result is stored as has_toc in the final metadata dictionary, allowing downstream processes to adjust parsing strategies for documents with explicit navigation structures.

Practical Implementation Examples

To analyze a document's structure programmatically:

from book_to_skill.utils import detect_structure

# Load a plain-text excerpt

with open("my_book.txt", "r", encoding="utf-8") as f:
    raw_text = f.read()

# Analyze document structure

info = detect_structure(raw_text)

print(f"Detected {info['chapters_detected']} chapters via {info['chapters_method']}")
if info['has_toc']:
    print("A Table-of-Contents was found.")
print("Sample headings:", info['chapter_headings_sample'])

The function returns a standardized dictionary:

{
  "chapters_detected": 12,
  "chapters_method": "numeric",
  "chapter_headings_sample": ["Chapter 1", "Chapter 2", "Chapter 3"],
  "has_toc": true
}

For testing individual lines against the chapter detection regexes:

from book_to_skill.utils import _chapter_number

print(_chapter_number("## Chapter 5: Introduction"))   # → 5

print(_chapter_number("第 ３ 回"))                     # → 3

print(_chapter_number("## i. introduction"))          # → 1 (Roman numeral)

print(_chapter_number("บทที่ ๕"))                    # → 5

Summary

• detect_structure() in book_to_skill/utils.py serves as the primary API for document structure analysis, implementing a two-phase detection cascade.

• Explicit chapter detection uses _chapter_number() with 10+ multilingual regex patterns covering Latin, Chinese, Thai, Hindi, Korean, and Persian scripts, plus Roman numerals.

• Structural fallback via _structural_chapter_count() analyzes Markdown/AsciiDoc heading hierarchies when explicit chapter markers are absent.

• False positive prevention includes code fence detection via _closed_fence_line_numbers(), digit-only title filtering, and punctuation validation.

• ToC detection scans document headers for multilingual table-of-contents patterns, storing results in the has_toc metadata field.

• Return format provides deterministic output including chapter count, detection method, sample headings, and ToC presence.

Frequently Asked Questions

How does book-to-skill handle documents with mixed languages?

The regex suite in _chapter_number() is designed to match chapter patterns independently of surrounding content. Since the patterns are tested sequentially against each line, a document containing both "Chapter 1" and "第2章" will correctly identify both chapters. The function extracts the numeric value regardless of the language script, storing distinct integers in the detection set.

What happens if a document uses only Markdown headings without explicit chapter numbers?

When detect_structure finds fewer than two explicit chapter numbers, it automatically invokes _structural_chapter_count(). This function analyzes ATX headings (# syntax) and Setext underlines (===) to find the shallowest heading level containing at least two titles. This structural approach effectively treats top-level Markdown sections as chapters without requiring explicit "Chapter N" text.

How does the library avoid detecting code examples as chapter headings?

The detection pipeline implements multiple guards. First, _closed_fence_line_numbers() identifies line ranges within Markdown code fences (triple backticks), and detect_structure skips these ranges during structural analysis. Additionally, titles containing only digits or punctuation without word characters (\w) are rejected, preventing code comments or decorative ASCII art from being classified as chapters.

Can I use the chapter detection logic independently of the full extraction pipeline?

Yes. The _chapter_number() function is available as a standalone utility in book_to_skill/utils.py for testing individual lines. For complete document analysis, detect_structure() provides the full metadata dictionary without requiring file I/O or skill generation steps, making it suitable for custom document preprocessing workflows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}