# Can book-to-skill Preserve Markdown Tables and Code Blocks from PDFs?

> Discover if book-to-skill can preserve Markdown tables and code blocks from PDFs. Learn how the default pipeline converts PDFs to plain text, losing formatting.

- Repository: [Virgilio Junior/book-to-skill](https://github.com/virgiliojr94/book-to-skill)
- Tags: deep-dive
- Published: 2026-08-30

---

**No, the standard book-to-skill pipeline converts PDFs to plain text using `pdftotext`, which flattens visual formatting including Markdown tables and fenced code blocks.**

The `book-to-skill` library extracts raw textual content from PDF documents to generate structured skill manifests, but it does not retain the original Markdown syntax or table layouts. Unless you implement custom post-processing logic, any tabular data or code blocks present in the source PDF will arrive as unformatted paragraphs in the final output.

## How book-to-skill Processes PDF Content

The extraction workflow treats PDFs as flat text streams rather than structured documents. In [`book_to_skill/parsers/pdf.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/pdf.py), the parser invokes the `pdftotext` utility to convert each page into a string of paragraphs. This text then flows through [`book_to_skill/sanitize.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/sanitize.py) for normalization—removing bidirectional control characters and standardizing line breaks—before reaching the skill generation logic orchestrated by [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py).

Because `pdftotext` extracts visible characters without preserving the semantic markup of the original document, Markdown delimiters such as pipe characters (`|`) for tables or triple backticks (`` ` ``) for code fences are not explicitly tagged. If the PDF rendered a table using visual spacing or monospaced fonts, the extraction tool sees only the raw words and whitespace, not the structural boundaries.

## Why Markdown Structure Is Lost

The core limitation lies in the **PDF-to-text conversion step**. When `pdftotext` processes a PDF, it strips away:

- **Column alignment and borders** that visually define tables
- **Font metadata** that might indicate code blocks (e.g., monospaced typefaces)
- **Indentation hierarchy** that distinguishes code from body text

The resulting string contains the words from your tables and code snippets, but without the surrounding Markdown syntax needed to reconstruct them. For example, a rendered table appears as space-separated values, and a code block becomes a series of indented lines indistinguishable from regular text until post-processing occurs.

## Implementing Custom Post-Processing

To preserve Markdown tables and code blocks from PDFs using book-to-skill, you must intercept the extracted text between the parser and sanitization stages. You can modify [`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py) or extend the pipeline in [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py) to apply pattern detection heuristics.

### Detecting and Reconstructing Markdown Tables

After `PdfParser.parse()` returns the raw string, scan for table-like patterns—rows containing multiple pipe characters (`|`) or columns separated by three or more spaces—and reformat them into valid Markdown table syntax with header separators.

```python
def reconstruct_markdown_tables(text: str) -> str:
    lines = text.splitlines()
    table_lines = []
    output = []
    
    for line in lines:
        # Detect pipe-separated rows or space-aligned columns

        if "|" in line or "   " in line and len(line.split()) >= 2:
            table_lines.append(line)
        else:
            if table_lines:
                # Convert accumulated lines to Markdown table

                header = table_lines[0].replace("  ", "|").strip()
                separator = "|".join(["---"] * len(header.split("|")))
                output.extend([header, separator] + table_lines[1:])
                table_lines = []
            output.append(line)
    
    return "\n".join(output)

```

### Recovering Fenced Code Blocks

Identify code sections by looking for consistent indentation (four or more spaces) or monospaced font patterns, then wrap them with triple backticks. Insert this logic after the PDF extraction but before the text reaches [`book_to_skill/sanitize.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/sanitize.py).

```python
import re

def fence_code_blocks(text: str) -> str:
    # Match blocks of 3+ consecutive lines starting with 4+ spaces

    pattern = r"(?:^ {4,}.+\n){3,}"
    matches = list(re.finditer(pattern, text, re.MULTILINE))
    
    # Replace from end to avoid offset issues

    for match in reversed(matches):
        block = match.group(0)
        # Remove leading indentation and wrap

        cleaned = "\n".join(line[4:] for line in block.rstrip().splitlines())
        fenced = f"```\n{cleaned}\n```\n"
        text = text[:match.start()] + fenced + text[match.end():]
    
    return text

```

## Key Source Files in book-to-skill

Understanding these specific implementation files helps when planning modifications:

- **[`book_to_skill/parsers/pdf.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/pdf.py)** — Contains the `pdftotext` invocation logic that converts PDF pages to strings.
- **[`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py)** — Entry point script that initializes the parser and handles raw extraction workflows.
- **[`book_to_skill/sanitize.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/sanitize.py)** — Applies generic text cleaning operations but does not restore structural formatting.
- **[`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py)** — Orchestrates the complete pipeline from PDF ingestion to skill manifest generation.

## Summary

- **book-to-skill** uses `pdftotext` utility in [`book_to_skill/parsers/pdf.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/pdf.py) to extract PDF content as plain text.
- The standard pipeline **flattens visual formatting**, discarding Markdown table syntax and code fence delimiters.
- Extracted tables appear as space-separated text; code blocks appear as indented paragraphs without language identifiers.
- You must implement **custom post-processing** between the parser and sanitization stages to reconstruct Markdown structures.
- Modify [`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py) or extend [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py) with pattern detection logic to recover tables and code blocks.

## Frequently Asked Questions

### Does book-to-skill support native PDF table extraction?

No, the library does not include native support for preserving table structures. According to the source code in [`book_to_skill/parsers/pdf.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/pdf.py), the tool relies entirely on `pdftotext`, which outputs linear text without cell boundaries or column metadata. You need custom logic to detect table patterns and regenerate Markdown pipe syntax.

### Can book-to-skill distinguish code blocks from regular text in PDFs?

By default, no. The extraction process in [`book_to_skill/parsers/pdf.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/pdf.py) treats all content as uniform text streams. While the PDF may contain font metadata indicating monospaced code sections, `pdftotext` does not expose this information. You must implement heuristics—such as detecting consistent indentation or specific keywords—to identify and fence code blocks during post-processing.

### Where should I add custom formatting logic in the book-to-skill pipeline?

Insert your reconstruction functions between the PDF parser and the sanitization module. Specifically, modify the workflow in [`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py) or create a preprocessor that runs after `PdfParser.parse()` but before the text reaches [`book_to_skill/sanitize.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/sanitize.py). This ensures you work with raw extracted text before normalization removes potentially useful whitespace patterns.

### Will future versions of book-to-skill automatically preserve Markdown formatting?

Based on the current architecture in [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py) and the reliance on external `pdftotext` calls, automatic preservation of Markdown structures would require either switching to a layout-aware PDF parser (such as `pdfplumber` or `PyMuPDF`) or adding a dedicated markup reconstruction layer. As of the current implementation, no such feature exists in the repository.