Chandra Image Extraction: How to Control Image Inclusion in OCR Output

Chandra always extracts page images via extract_images() in chandra/output.py, while the include_images flag determines whether those images appear in the generated HTML and Markdown.

The datalab-to/chandra OCR pipeline processes page-level layouts to produce structured text and visual outputs. Understanding how image extraction and inclusion work allows you to generate lightweight text-only exports or preserve visual elements for downstream validation. This guide examines the source code to demonstrate how cropped images are produced and how to toggle their appearance in the final results.

How Chandra Extracts Images from LLM Layouts

When processing a page, the underlying model returns raw HTML encoding the visual layout. Each Image or Figure block contains an <img> tag without a src attribute, representing a region of the original page that must be cropped and saved.

The extract_images() Implementation

The extract_images() function in chandra/output.py iterates over layout chunks, crops the corresponding regions from the original page image, and stores them as PIL.Image objects.

# chandra/output.py

def extract_images(html: str, chunks: dict, image: Image.Image):
    images = {}
    div_idx = 0
    for idx, chunk in enumerate(chunks):
        div_idx += 1
        if chunk["label"] in ["Image", "Figure"]:
            img = BeautifulSoup(chunk["content"], "html.parser").find("img")
            if not img:
                continue
            bbox = chunk["bbox"]
            try:
                block_image = image.crop(bbox)          # ← crop original page image

            except ValueError:
                continue
            img_name = get_image_name(html, div_idx)   # deterministic filename

            images[img_name] = block_image
    return images

The function generates deterministic filenames using get_image_name(), typically formatted as <hash>_<idx>_img.webp, and returns a dictionary mapping these names to the cropped image objects. This extraction step runs unconditionally regardless of output preferences, ensuring the visual data is always available for programmatic use.

Controlling Image Inclusion in HTML and Markdown

While extraction is mandatory, inclusion of image references in the textual output is optional. Chandra provides the include_images boolean flag to filter visual blocks from the final HTML and Markdown.

The include_images Flag

Both parse_html() and parse_markdown() in chandra/output.py accept an include_images parameter that defaults to True.

# chandra/output.py – parse_html()

def parse_html(html: str, include_headers_footers: bool = False,
               include_images: bool = True):
    …
    if label and not include_images:
        if label in ["Image", "Figure"]:
            continue                      # skip image blocks entirely

    …
    if label in ["Image", "Figure"]:
        img = div.find("img")
        img_src = get_image_name(html, div_idx)
        if img:
            img["src"] = img_src        # inject deterministic src attribute

        else:
            img = BeautifulSoup(f"<img src='{img_src}'/>", "html.parser")
            div.append(img)

When include_images=False, the function skips all div blocks labeled Image or Figure, ensuring they never appear in the output. When True, the blocks remain and receive deterministic src attributes pointing to the extracted filenames. The parse_markdown() function calls parse_html() with the same flag before converting the result to Markdown via a custom Markdownify converter.

How the Flag Propagates Through the Stack

The include_images setting travels through multiple layers of the Chandra architecture:

• CLI Layer: chandra/scripts/cli.py defines --include-images / --no-images (default True) using Click, passing the value to model.generate().
• InferenceManager: Located in chandra/model/__init__.py, this class extracts include_images from **kwargs and stores it in output_kwargs.
• Post-Processing: The manager calls parse_markdown() and parse_html() with these kwargs, but runs extract_images() always, independent of the flag.
• Result Object: The BatchOutputItem contains three fields—markdown, html, and images—meaning the cropped images remain accessible even when excluded from the text.

Practical Usage Examples

You can control image extraction behavior via command-line flags or programmatically through the Python API.

Excluding Images via CLI

Run the CLI with the --no-images flag to generate Markdown and HTML files without <img> tags:

chandra-cli input_folder/ output_folder/ --no-images

Omitting this flag (or using --include-images) preserves image references in the output.

Programmatic Control with InferenceManager

When using the Python API, pass include_images=False to InferenceManager.generate() to suppress image tags while retaining access to the raw crops:

from pathlib import Path
from chandra.model import InferenceManager
from chandra.input import load_file
from chandra.model.schema import BatchInputItem

# Load a single page image

images = load_file("sample_page.png", {})
batch = [BatchInputItem(image=img, prompt_type="ocr_layout") for img in images]

# Disable image tags in markdown/html

manager = InferenceManager(method="vllm")
results = manager.generate(batch, include_images=False)

# Access markdown (no <img> tags)

print(results[0].markdown)

# Access the extracted cropped images (still available)

for name, pil_img in results[0].images.items():
    pil_img.save(Path("extracted") / name)   # manual saving

Embedding Images for Web Display

For applications requiring inline images, such as the Streamlit demo in chandra/scripts/app.py, convert extracted images to base64 data URLs using the helper function:

from chandra.scripts.app import embed_images_in_markdown

# `result` is a BatchOutputItem from the manager

markdown_with_images = embed_images_in_markdown(result.markdown, result.images)

# Renders images as base64 data URLs within the markdown

Summary

• Extraction is unconditional: extract_images() in chandra/output.py always runs, producing a dictionary of cropped PIL.Image objects keyed by deterministic filenames.
• Inclusion is configurable: The include_images boolean (CLI: --include-images/--no-images) controls whether image blocks appear in the HTML and Markdown output.
• Dual access: The BatchOutputItem provides separate access to text outputs and the raw images dictionary, enabling workflows that omit visual references from text while preserving images for separate processing.

Frequently Asked Questions

Does Chandra always extract images even when they are excluded from the text output?

Yes. According to the source code in chandra/output.py, the extract_images() function executes regardless of the include_images flag. The flag only controls whether parse_html() and parse_markdown() include the image blocks in the generated text, allowing you to omit visual references from Markdown while still accessing the cropped images via the BatchOutputItem.images dictionary.

What filename format does Chandra use for extracted images?

Chandra generates deterministic filenames using the get_image_name() helper function, typically formatted as <hash>_<idx>_img.webp. This naming convention ensures consistent references between the extracted image dictionary and the src attributes injected into HTML during the parsing phase.

How can I embed extracted images directly into Markdown for web display?

Use the embed_images_in_markdown() function from chandra/scripts/app.py. This utility converts the PIL.Image objects in the result's images dictionary to base64 data URLs and rewrites the Markdown image references accordingly, producing self-contained output suitable for rendering in browsers or Streamlit applications without external file dependencies.

Where is the include_images flag defined in the Chandra CLI?

The flag is defined in chandra/scripts/cli.py using the Click library, which exposes --include-images (default True) and its negation --no-images. This value is passed through to the InferenceManager.generate() method and subsequently propagated to the post-processing functions in chandra/output.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}