# Supported Prompt Types in Chandra: How PROMPT_MAPPING Configures OCR Modes

> Explore supported prompt types in Chandra: ocr_layout and ocr. Learn how PROMPT_MAPPING in prompts.py configures LLM inference for OCR modes.

- Repository: [Datalab/chandra](https://github.com/datalab-to/chandra)
- Tags: how-to-guide
- Published: 2026-03-27

---

**Chandra supports two prompt types—`ocr_layout` and `ocr`—defined in [`chandra/prompts.py`](https://github.com/datalab-to/chandra/blob/main/chandra/prompts.py) where the `PROMPT_MAPPING` dictionary maps each identifier to a specific prompt template for LLM inference.**

The `datalab-to/chandra` repository provides an OCR framework that uses large language models to convert images to HTML. Understanding the supported prompt types in Chandra and how `PROMPT_MAPPING` is configured is essential for selecting the correct inference mode for your use case, whether you need simple text extraction or layout-aware HTML generation.

## Supported Prompt Types in Chandra

Chandra currently defines two supported prompt identifiers in the `PROMPT_MAPPING` dictionary:

- **`ocr_layout`**: Performs OCR with layout analysis, returning HTML `<div>` blocks that include bounding-box metadata for precise document structure preservation.
- **`ocr`**: Performs plain OCR conversion, transforming an image directly into HTML markup without explicit layout metadata.

These identifiers are the only valid keys accepted by the inference backends when constructing a `BatchInputItem`.

## How PROMPT_MAPPING Is Configured

The mapping between prompt identifiers and their corresponding templates lives in [`chandra/prompts.py`](https://github.com/datalab-to/chandra/blob/main/chandra/prompts.py). According to the datalab-to/chandra source code, the configuration is implemented as a simple dictionary that references prompt strings defined earlier in the same module:

```python

# chandra/prompts.py

PROMPT_MAPPING = {
    "ocr_layout": OCR_LAYOUT_PROMPT,
    "ocr": OCR_PROMPT,
}

```

Both `OCR_LAYOUT_PROMPT` and `OCR_PROMPT` are string templates defined immediately above the mapping in [`chandra/prompts.py`](https://github.com/datalab-to/chandra/blob/main/chandra/prompts.py). These templates contain the specific instructions and formatting requirements sent to the underlying language model, including allowed HTML tags and attributes for the output structure.

## How Inference Backends Use PROMPT_MAPPING

The `PROMPT_MAPPING` dictionary serves as the single source of truth for prompt resolution across Chandra's model implementations.

### VLLM Backend ([`chandra/model/vllm.py`](https://github.com/datalab-to/chandra/blob/main/chandra/model/vllm.py))

In the VLLM inference backend, the system retrieves the appropriate prompt text by indexing into `PROMPT_MAPPING` using the `prompt_type` attribute of the input item:

```python
prompt = PROMPT_MAPPING[item.prompt_type]

```

This lookup occurs during the request preparation phase before sending the image and prompt to the VLLM engine.

### HuggingFace Backend ([`chandra/model/hf.py`](https://github.com/datalab-to/chandra/blob/main/chandra/model/hf.py))

Similarly, the HuggingFace backend resolves prompt strings using the same dictionary:

```python
prompt = PROMPT_MAPPING[prompt_type]

```

This ensures consistent prompt behavior regardless of which backend executes the inference.

## Practical Examples: Selecting Prompt Types

When building an inference request, you explicitly set the `prompt_type` parameter when constructing a `BatchInputItem`. Here are the two supported configurations:

### Layout-Aware OCR Request

To extract text while preserving document layout information, use the `ocr_layout` prompt type as implemented in [`chandra/scripts/screenshot_app.py`](https://github.com/datalab-to/chandra/blob/main/chandra/scripts/screenshot_app.py):

```python
from chandra.input import BatchInputItem

# img is a PIL Image or similar

item = BatchInputItem(
    image=img,
    prompt_type="ocr_layout",  # Maps to OCR_LAYOUT_PROMPT

)

result = model.predict(item)

```

### Standard OCR Request

For straightforward image-to-HTML conversion without layout metadata, specify the `ocr` prompt type:

```python
from chandra.input import BatchInputItem

item = BatchInputItem(
    image=img,
    prompt_type="ocr",  # Maps to OCR_PROMPT

)

result = model.predict(item)

```

Both examples appear throughout the codebase, including in [`chandra/scripts/cli.py`](https://github.com/datalab-to/chandra/blob/main/chandra/scripts/cli.py) and the integration tests in [`tests/integration/test_image_inference.py`](https://github.com/datalab-to/chandra/blob/main/tests/integration/test_image_inference.py).

## Summary

- Chandra supports exactly two prompt types: **`ocr_layout`** for layout-aware HTML generation and **`ocr`** for plain text extraction.
- The **`PROMPT_MAPPING`** dictionary in [`chandra/prompts.py`](https://github.com/datalab-to/chandra/blob/main/chandra/prompts.py) maps these identifiers to their respective prompt templates (`OCR_LAYOUT_PROMPT` and `OCR_PROMPT`).
- Both the VLLM backend ([`chandra/model/vllm.py`](https://github.com/datalab-to/chandra/blob/main/chandra/model/vllm.py)) and HuggingFace backend ([`chandra/model/hf.py`](https://github.com/datalab-to/chandra/blob/main/chandra/model/hf.py)) resolve prompts using `PROMPT_MAPPING[item.prompt_type]`.
- You select the prompt type by setting the `prompt_type` parameter when creating a `BatchInputItem` instance.

## Frequently Asked Questions

### What prompt types does Chandra currently support?

Chandra supports two prompt types: `ocr_layout` for OCR with layout analysis (returning HTML with bounding-box metadata) and `ocr` for plain OCR (returning HTML without explicit layout information). These are defined in the `PROMPT_MAPPING` dictionary in [`chandra/prompts.py`](https://github.com/datalab-to/chandra/blob/main/chandra/prompts.py).

### Where is the PROMPT_MAPPING dictionary defined?

The `PROMPT_MAPPING` dictionary is defined in [`chandra/prompts.py`](https://github.com/datalab-to/chandra/blob/main/chandra/prompts.py) at the module level. It maps string identifiers like `"ocr_layout"` and `"ocr"` to their corresponding prompt template variables (`OCR_LAYOUT_PROMPT` and `OCR_PROMPT`) that contain the actual LLM instructions.

### How do I select a prompt type when using Chandra?

You select a prompt type by passing the identifier string to the `prompt_type` parameter when constructing a `BatchInputItem`. For example: `BatchInputItem(image=img, prompt_type="ocr_layout")`. The inference backend then uses this key to look up the full prompt template in `PROMPT_MAPPING`.

### Can I add custom prompt types to Chandra?

Currently, Chandra only supports the two predefined prompt types defined in [`chandra/prompts.py`](https://github.com/datalab-to/chandra/blob/main/chandra/prompts.py). To add custom prompt types, you would need to modify the source code to add your new prompt template variable to the file and register it in the `PROMPT_MAPPING` dictionary with a new unique key.