How book-to-skill Handles Chapter Heading Formats: Roman Numerals, ATX Markdown, and Multilingual Support

book-to-skill employs a hierarchical detection system that prioritizes numeric chapter identifiers (including Roman numerals) via regex matching in tools/discovery_tax.py, falling back to ATX Markdown headings only when explicit numbering is absent.

The virgiliojr94/book-to-skill repository provides a robust parser for extracting structured skills from book content, requiring sophisticated chapter boundary detection across diverse formatting conventions. Understanding how this tool handles different heading formats—from classic Roman numerals to standard ATX Markdown—reveals the flexibility of its _structural_chapter_count implementation.

The Hierarchical Detection Strategy

The chapter detection logic operates through a prioritized two-tier system implemented in tools/discovery_tax.py. This approach ensures maximum compatibility with both explicitly numbered chapters and generically formatted Markdown documents.

Numeric and Roman Numeral Recognition

The primary detection mechanism uses a comprehensive regular expression defined in tools/discovery_tax.py to identify leading numeric tokens:

RE_NUM = re.compile(
    r"(?i)^(?:chapter\s+|章节\s+|فصل\s+)?"
    r"(?P<num>(?:[ivxlcdm]+|\d+))"  # Roman or Arabic

    r"[\s.:-]*"
)

This pattern captures:

• Arabic numerals: 1, 02, 15
• Roman numerals: I, II, III, IV (case-insensitive)
• Multilingual prefixes: "Chapter", "章" (Chinese), "فصل" (Persian/Arabic), and other language-specific chapter markers

When matched, Roman numerals undergo normalization via roman.fromRoman, converting identifiers like IV or XII into their Arabic equivalents for consistent internal storage.

ATX Markdown Headings as Fallback

When _structural_chapter_count fails to locate numeric headings, the system falls back to ATX-style Markdown headings—lines beginning with one or more # characters. This fallback mechanism activates only after the numeric scan returns empty, ensuring that explicit chapter numbering takes precedence over structural Markdown hierarchy.

The ATX detection specifically excludes headings embedded within fenced code blocks (validated by test_unbalanced_code_fence.py), preventing false positives from commented code or documentation examples.

Setext Style and Punctuation Guards

Beyond numeric and ATX formats, the parser recognizes Setext-style headings—text lines followed by underline characters (=, -, or ~). However, the implementation includes a critical guard clause exercised in test_setext_punctuation_guard.py:

Pure punctuation underlines (such as *** followed by ---) are classified as thematic breaks rather than headings. Only underlines containing non-punctuation content trigger chapter detection, distinguishing true structural headings from horizontal rules or decorative separators.

Multilingual and Language-Specific Detection

The regex architecture supports extensive multilingual chapter identification beyond standard English conventions. As verified in test_book_to_skill.py, the system handles:

• Hindi: Devanagari chapter markers with numeric suffixes
• Korean: Hangul chapter indicators (예: 제1장)
• Persian/Bengali/Thai: Language-specific words for "chapter" followed by numeric tokens

This allows mixed-script headings such as 章节Ⅰ (Chinese chapter marker with Roman numeral) or فصل ۲ (Persian chapter marker with Eastern Arabic numeral) to parse correctly, with the prefix stripped before numeric validation.

Handling Edge Cases and Structural Integrity

Fenced Code Block Filtering

Before heading detection begins, book_to_skill/parsers/text.py pre-processes the source text through read_text_file, which includes logic to ignore lines contained within triple-backtick fenced code blocks. The test suite confirms this behavior in test_unbalanced_code_fence.py, ensuring that Markdown documentation within books does not generate spurious chapter boundaries.

Deduplication Logic

When both a Table of Contents entry and a body heading reference the same numeric identifier (for example, "Chapter I" appearing in both locations), the system applies deduplication rules verified in test_chapter_method_reported.py. Only one chapter instance is counted, preventing double-counting of the same structural boundary.

Implementation Workflow

The complete detection pipeline flows through these specific components:

1. File Ingestion: read_text_file in book_to_skill/parsers/text.py reads and normalizes text encodings
2. Code Filtering: Fenced code blocks are identified and excluded from heading scans
3. Numeric Scan: _structural_chapter_count executes the Roman/Arabic regex against remaining lines
4. Fallback Processing: If no numeric matches exist, ATX heading counting commences
5. Normalization: Roman numerals convert to Arabic equivalents; multilingual prefixes are stripped

# Example: Mixed format recognition

from book_to_skill import extract

result = extract.from_path("mixed_chapters.md")
print(result["chapter_headings_sample"])

# Output: ['I', 'II', 'III'] 

# (Recognizing Roman numerals regardless of ATX or Setext formatting)

Summary

• Primary Detection: Numeric regex in tools/discovery_tax.py handles Roman numerals, Arabic numbers, and multilingual prefixes through case-insensitive pattern matching.
• ATX Fallback: Markdown # headings serve as secondary detection when no numbered chapters exist, with explicit exclusion of code block contents.
• Setext Support: Underlined headings are recognized only when underlines contain non-punctuation characters, preventing confusion with horizontal rules.
• Multilingual Capability: The system accommodates Hindi, Korean, Persian, Bengali, Thai, and other scripts through language-specific word matching.
• Integrity Guards: Deduplication logic and fenced-code filtering ensure accurate chapter counts in complex documents.

Frequently Asked Questions

How does book-to-skill differentiate between Roman numerals and regular text?

The tool uses a specific regex pattern (?:[ivxlcdm]+|\d+) within tools/discovery_tax.py that matches valid Roman numeral combinations (I, V, X, L, C, D, M) alongside Arabic digits. Valid Roman numerals are then converted to Arabic integers using roman.fromRoman for consistent chapter indexing, while invalid letter combinations are ignored.

Will ATX headings inside code blocks be counted as chapters?

No. The parser explicitly filters out content within fenced code blocks (```) before scanning for headings. This behavior is validated by test_unbalanced_code_fence.py, ensuring that Markdown examples or code comments containing # characters do not create false chapter boundaries.

Can book-to-skill handle chapters written in non-Latin scripts?

Yes. The detection regex includes support for multilingual chapter prefixes such as "章节" (Chinese), "فصل" (Persian/Arabic), and markers in Hindi, Korean, Bengali, and Thai. Mixed-script headings like 第Ⅰ章 are parsed by stripping the script-specific word and processing the remaining Roman or Arabic numeral.

What happens if a document uses both Roman numerals and ATX headings?

The hierarchical detection in _structural_chapter_count prioritizes explicit numeric identifiers. If Roman numerals (or Arabic numbers) are detected anywhere in the document, those become the authoritative chapter markers. ATX headings only activate as a fallback mechanism when no numeric chapter indicators are found, preventing format mixing within a single output.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}