# How book-to-skill Detects Chapters in Multiple Languages: Pattern Matching Across Scripts

> Discover how book-to-skill detects chapters in multiple languages by leveraging pattern matching across diverse scripts and dedicated regex matchers for over ten languages.

- Repository: [Virgilio Junior/book-to-skill](https://github.com/virgiliojr94/book-to-skill)
- Tags: how-to-guide
- Published: 2026-08-28

---

**book-to-skill determines chapter boundaries by scanning extracted plain text for language-specific numeric patterns, using dedicated regex matchers that handle Arabic numerals, Roman numerals, CJK ideographs, and script-specific chapter words across more than ten languages.**

The `book-to-skill` repository processes documents in diverse languages to extract structured content for skill generation. To accurately **detect chapters in multiple languages**, the project implements a sophisticated two-stage pattern matching system in [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py) that recognizes culturally distinct heading conventions—from German *Kapitel* to Chinese *第N章*—without requiring external language detection libraries.

## The Two-Stage Detection Pipeline

The detection algorithm operates through a fast filtering stage followed by aggregation logic that validates structural integrity.

### Stage 1: Fast Numeric Pattern Matching with `_chapter_number()`

The function `_chapter_number()` (located at [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py) lines 62‑73) serves as the primary entry point for heading analysis. It first attempts a fast "plain numbered" pattern (e.g., `1  Introduction`) for efficiency. When this fails, it delegates to `_match_chapter_number()`, which contains the language-specific regular expressions for deeper inspection.

### Stage 2: Distinct Number Aggregation in `detect_structure()`

The `detect_structure()` function (lines 91‑99) iterates over every line of text, invoking `_chapter_number()` to extract numeric values. It collects **distinct** numeric values to determine the chapter count. The count of unique numbers becomes the *numeric* chapter count; if no numeric headings are identified, the system falls back to a *structural* algorithm implemented in `_structural_chapter_count()`.

## Language-Specific Regex Patterns

The multilingual capability derives from explicit regex definitions that capture localized chapter naming conventions.

### Latin Script Languages: `_EXPLICIT_CHAPTER`

For **English, French, German, Italian, Dutch, Spanish, and Portuguese**, the `_EXPLICIT_CHAPTER` pattern (lines 103‑111) matches explicit chapter words followed by Arabic or Roman numerals. The pattern recognizes *chapter*, *chapitre*, *kapitel*, *capítulo*, *capitolo*, and *hoofdstuk*. The `_HEADING_TAIL` validation ensures matches represent actual headings rather than prose mentions.

### Roman Numeral Support: `_ROMAN_HEAD` and `_LC_MD_ROMAN`

The system handles classical enumeration through `_ROMAN_HEAD` and `_LC_MD_ROMAN` (lines 118‑124), matching headings such as `I: Introduction` or `## i. Preface`. Captured numerals convert to integers via `_roman_to_int()`.

### CJK Languages: Chinese and Japanese

Two distinct patterns handle Simplified/Traditional Chinese and Japanese:

- `_CN_CHAPTER` (lines 132‑140): Matches `<第 N 章>` style headings using full-width Arabic digits and CJK ideographs.
- `_MD_CN_HEADING`: Recognizes markdown formats like `## 一 · Title`.

Pre-processing via `_KANGXI_NUMERAL_TRANS` normalizes Kangxi radicals to standard ideographs before matching.

### Thai Chapter Detection: `_TH_CHAPTER`

The `_TH_CHAPTER` pattern (lines 167‑174) identifies headings such as `บทที่ 3` or `ตอนที่ ๑`. Thai digits map to Arabic equivalents before integer conversion.

### Hindi (Devanagari): `_HI_CHAPTER`

For Hindi documents, `_HI_CHAPTER` (lines 77‑84) detects `अध्याय 1` or `अध्याय १`. The system translates Devanagari digits to ASCII before numerical processing.

### Korean Chapter Patterns: `_KO_CHAPTER`

The `_KO_CHAPTER` regex (lines 89‑102) captures Korean statutes and books using `제1장`, `제4장`, or `제2편`. It optionally handles "의 N" suffixes and relies on classifiers like `장|편|절|관`.

### Persian (Farsi): `_fa_chapter_number()`

A dedicated pipeline (lines 64‑85) handles Persian documents, parsing both digit forms (`فصل ۱`) and word ordinals (`فصل بیست و یکم`). It additionally recognizes `بخش ۲: …` structures.

## Algorithm Optimizations and Guards

Beyond pattern matching, the system implements defensive programming to avoid false positives.

### Pre-processing and Normalization

Before regex evaluation, `_KANGXI_NUMERAL_TRANS` normalizes archaic CJK numerals. Lines exceeding 80 characters are ignored via length guards to prevent false positives in OCR-generated noise.

### Markdown and AsciiDoc Handling

When lines begin with heading markers (`##`, `===`, etc.), the `_MD_HEADING_PREFIX` logic strips these prefixes and retries matching. This enables recognition of formatted headings like `## Chapter 1` across all supported languages.

### Structural Fallback Analysis

If **no** numeric headings are discovered, `_structural_chapter_count()` analyzes the Markdown/AsciiDoc hierarchy (ATX, Setext, and fenced-code exclusion) to infer chapter-like divisions (referenced around lines 83‑101).

## Working with the Detection API

The `detect_structure()` function returns a standardized metadata dictionary:

```python
from book_to_skill.utils import detect_structure

# Mixed English and Chinese content

text = """

# Introduction

Chapter 1  The Beginning

## 第 2 章 章节标题

Chapter 3  Advanced Topics
"""

result = detect_structure(text)
print(result["chapters_detected"])   # → 3

print(result["chapters_method"])     # → "numeric"

print(result["chapter_headings_sample"])

# → ['Chapter 1  The Beginning', '## 第 2 章 章节标题', 'Chapter 3  Advanced Topics']

```

For direct testing of individual lines, use the low-level matcher:

```python
from book_to_skill.utils import _chapter_number

print(_chapter_number("บทที่ 3"))        # → 3  (Thai)

print(_chapter_number("अध्याय ५"))        # → 5  (Hindi)

print(_chapter_number("第 ７ 章"))       # → 7  (Chinese)

print(_chapter_number("Kapitel 12"))    # → 12 (German)

```

The resulting metadata attaches to each extracted source and consolidates into [`metadata.json`](https://github.com/virgiliojr94/book-to-skill/blob/main/metadata.json) with fields `chapters_detected`, `chapters_method` (values: `"numeric"`, `"structural"`, or `"none"`), `chapter_headings_sample`, and `has_toc`.

## Summary

- **book-to-skill** detects chapters through a two-stage pipeline: fast numeric extraction via `_chapter_number()` and validation via `detect_structure()`.
- The system supports **10+ languages** through dedicated regex patterns in [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py), including CJK, Devanagari, Thai, and Persian scripts.
- **Roman numerals** and **Arabic digits** are normalized across all supported writing systems.
- **Length guards** (80-character limit) and **markdown prefix stripping** prevent false positives in structured documents.
- A **structural fallback** analyzes document hierarchy when numeric patterns fail.
- Results expose both the detection method and sample headings to downstream skill generation pipelines.

## Frequently Asked Questions

### Which languages does book-to-skill support for chapter detection?

The library explicitly supports English, French, German, Italian, Dutch, Spanish, Portuguese, Chinese (Simplified/Traditional), Japanese, Thai, Hindi, Korean, and Persian. Each language implements specific regex patterns in [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py) to recognize localized chapter terminology such as *第*, *अध्याय*, *บทที่*, and *فصل*.

### How does the system handle Roman numerals in chapter headings?

The `_ROMAN_HEAD` and `_LC_MD_ROMAN` patterns (lines 118‑124) capture Roman numeral prefixes in both uppercase and lowercase formats. Captured values convert to integers via `_roman_to_int()`, allowing the system to count chapters consistently regardless of whether the source uses Arabic or Roman enumeration.

### What happens when no numeric chapter patterns are found?

When `detect_structure()` cannot identify numeric headings, it automatically invokes `_structural_chapter_count()`. This fallback analyzes Markdown/AsciiDoc structural elements—specifically ATX and Setext heading hierarchies—excluding fenced code blocks to infer logical chapter boundaries. The resulting `chapters_method` field reports `"structural"` instead of `"numeric"`.

### Can the detection handle mixed-language documents?

Yes. The `_chapter_number()` function evaluates each line independently against all language patterns. A document containing English "Chapter 1" followed by Chinese "第 2 章" correctly registers as containing two distinct chapters, as demonstrated in the `chapter_headings_sample` output array.