# How book-to-skill Detects Chapters in Different Languages: A Technical Deep Dive

> Explore how book-to-skill detects chapters across multiple languages using regex and Markdown headings. Understand the technical details behind multilingual chapter identification.

- Repository: [Virgilio Junior/book-to-skill](https://github.com/virgiliojr94/book-to-skill)
- Tags: deep-dive
- Published: 2026-08-30

---

**book-to-skill detects chapters in different languages using a cascade of language-specific regex patterns in the `_match_chapter_number` function, supporting everything from English "Chapter 1" to Chinese "第一章" and Hindi "अध्याय १", with a fallback to structural Markdown heading detection when numeric patterns fail.**

The open-source repository `virgiliojr94/book-to-skill` extracts structured content from digital books regardless of the source language. Its chapter detection system relies on [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py), which contains specialized regular expressions for over ten major writing systems. This multilingual approach ensures the tool can segment books into logical sections even when processing mixed-script documents or non-Latin alphabets.

## The Two-Pass Detection Strategy

The chapter detection algorithm operates in two sequential passes when analyzing text. This approach maximizes accuracy across different formatting conventions while maintaining processing speed.

### Pure-Text Pattern Matching

During the first pass, the `_chapter_number` function scans each line for chapter headings that begin the line or follow standard Markdown/AsciiDoc prefixes. The system checks against a prioritized list of language-specific regex patterns implemented in `_match_chapter_number`. If a line matches any pattern, the function extracts the numeric identifier—either as Arabic numerals, Roman numerals, or language-specific number words—and returns it as an integer.

### Markdown Prefix Fallback

If the initial pure-text scan fails, the algorithm strips common Markdown heading markers such as `##` or AsciiDoc style `==` prefixes and retries the match. This fallback mechanism, located in `_chapter_number` (lines 378-405 of [`utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/utils.py)), ensures that formatted documents do not bypass detection simply because of syntactic markup.

## Language-Specific Regex Patterns

The heart of the detection system lives in the regex definitions spanning lines 103-258 of [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py). Each pattern targets specific linguistic conventions for marking chapter divisions.

### Latin Script Languages

For English, Spanish, Portuguese, French, German, Italian, Dutch, and Vietnamese, the system uses a comprehensive pattern matching terms like "chapter", "chapitre", "capítulo", "hoofdstuk", and "chương". The regex at lines 133-176 captures both Arabic numerals (`\d{1,2}`) and Roman numerals (`[IVXLCDMivxlcdm]{1,7}`), allowing detection of headings like "Chapter 5: Introduction", "Capítulo II. El principio", or "Ch. 10".

### CJK Scripts

Chinese support handles both simplified and traditional characters through patterns recognizing "第 N 章/回/卷/节/篇/讲" structures. The regex at lines 185-200 captures Chinese numerals including "零一二两三四五六七八九十百千" and full-width digits. A separate Markdown-aware pattern (lines 201-214) handles prefixed headings like `## 第 一讲 · 概论`.

Korean detection (lines 250-258) identifies "제 N 장/편/절/관" formats, matching headings such as "제 4장 근로시간과 휴식" while accounting for optional ordinal suffixes.

### Indic Scripts

Hindi and Bengali implementations use Devanagari and Bengali script recognition respectively. The Hindi pattern at lines 217-225 matches "अध्याय N" (e.g., "अध्याय १"), while the Bengali equivalent at lines 227-236 detects "অধ্যায় N" (e.g., "অধ্যায় ৩"). Both patterns include optional Markdown prefix handling and language-specific digit ranges (`[०-९]` for Hindi, `[০-৯]` for Bengali).

### Thai and Persian Support

Thai detection (lines 239-247) recognizes "บทที่ N", "ตอนที่ N", and "ภาคที่ N" formats using Thai digits (`[0-9๐-๙]+`). Persian support implements complex logic in the separate `_fa_chapter_number` function to handle Arabic-Indic digits, Persian ordinals, and glued PDF text artifacts commonly found in Persian publications.

## The Core Detection Algorithm

The `detect_structure` function (lines 608-639 of [`utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/utils.py)) orchestrates the entire process. It iterates through document lines, collecting distinct chapter numbers and sample headings:

```python
def detect_structure(text: str) -> dict:
    numbers = set()
    headings = []
    
    for line in text.splitlines():
        num = _chapter_number(line)          # tries all language patterns

        if num is not None:
            numbers.add(num)                 # distinct chapter numbers only

            headings.append(line.strip())
    
    # Determine detection method

    if len(numbers) >= 2:
        method = "numeric"
    else:
        method = "structural"
        chapters_detected = _structural_chapter_count(text)
    
    return {
        'chapters_detected': len(numbers) if method == "numeric" else chapters_detected,
        'chapters_method': method,
        'chapter_headings_sample': headings[:5],
        'has_toc': False
    }

```

When fewer than two distinct numeric headings are found, the system falls back to `_structural_chapter_count`, which counts generic Markdown/AsciiDoc headings without requiring specific chapter keywords.

## Implementation Reference Points

All detection logic resides in [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py) with the following key locations:

- **Chapter-type regexes**: Lines 103-112, 133-176, 185-200, 201-214, 217-225, 227-236, 239-247, and 250-258
- **`_match_chapter_number`**: Lines 221-275
- **`_chapter_number`** (Markdown prefix handling): Lines 378-405
- **`detect_structure`** (overall orchestration): Lines 608-639

The parsing pipeline begins in [`book_to_skill/parsers/text.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/text.py), which provides the `read_text_file` helper used before structure detection. Unit tests in [`tests/test_chapter_method_reported.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tests/test_chapter_method_reported.py) verify multilingual detection accuracy across supported scripts.

## Practical Usage Examples

The `detect_structure` function accepts raw text strings and returns detection metadata:

```python
from book_to_skill.utils import detect_structure

# English example

english = """\
Chapter 1: Introduction
...
Chapter 2: Methods
"""
print(detect_structure(english)['chapters_detected'])

# → 2 (numeric)

# Spanish example

spanish = """\
Capítulo 1 – Visión general
...
Capítulo 2 – Detalles técnicos
"""
print(detect_structure(spanish)['chapters_detected'])

# → 2 (numeric)

# Chinese example

chinese = """\
第 一章 起航
...
第 二章 航行
"""
print(detect_structure(chinese)['chapters_detected'])

# → 2 (numeric)

# Thai example

thai = """\
บทที่ 1 การเริ่มต้น
...
บทที่ 2 การทำงานต่อ
"""
print(detect_structure(thai)['chapters_detected'])

# → 2 (numeric)

# Korean example

korean = """\
제 1장 서론
...
제 2장 본문
"""
print(detect_structure(korean)['chapters_detected'])

# → 2 (numeric)

```

Each call returns a dictionary containing `chapters_detected` (integer count), `chapters_method` (either "numeric" or "structural"), `chapter_headings_sample` (list of up to 5 heading strings), and `has_toc` (boolean).

## Summary

- **book-to-skill** detects chapters in different languages using regex patterns tailored to specific scripts and linguistic conventions in [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py).
- The **two-pass algorithm** first attempts pure-text matching, then strips Markdown prefixes and retries if initial detection fails.
- **Supported scripts** include Latin (English, Spanish, French, etc.), CJK (Chinese, Korean), Indic (Hindi, Bengali), Thai, and Persian, each with dedicated regex patterns handling both Arabic and native numerals.
- The **`detect_structure`** function returns detection metadata including method type ("numeric" vs "structural") and sample headings.
- A **structural fallback** counts generic Markdown headings when fewer than two numeric chapter headings are identified.

## Frequently Asked Questions

### How does book-to-skill handle books with mixed languages?

The detection system processes each line independently against all language patterns simultaneously. Since patterns for different scripts use mutually exclusive Unicode ranges, a document containing both English "Chapter 1" and Chinese "第二章" will detect chapters from both languages, though the `chapters_method` classification depends on the dominant pattern found.

### What happens if a book uses Roman numerals instead of Arabic numbers?

The Latin script regex explicitly captures Roman numerals via the `(?P<roman>[IVXLCDMivxlcdm]{1,7})` named group. Headings like "Chapter IV" or "Part II. The Beginning" match successfully and convert to integers (4 and 2 respectively) for the numeric detection method.

### Can the system detect chapters in PDF files with corrupted text encoding?

The Persian-specific `_fa_chapter_number` function includes logic to handle "glued PDF text" artifacts where spaces between words have been lost during extraction. For other languages, the regex patterns use word boundaries (`\b`) and flexible whitespace matching (`\s*`) to accommodate minor encoding irregularities.

### Where is the chapter detection logic located in the repository?

All multilingual chapter detection code resides in [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py) between lines 103-405, with the primary orchestration function `detect_structure` located at lines 608-639. The regex patterns span lines 103-258, while helper functions like `_chapter_number` and `_match_chapter_number` occupy lines 221-275 and 378-405 respectively.