How to Use Claude with Vision for Chart and Graph Interpretation

Claude's vision model can analyze PDFs containing charts and graphs by encoding them as Base64 documents and sending them to the Anthropic API with a special beta header, allowing the model to read visual elements like axes, legends, and data points that pure text extraction would miss.

The anthropic/claude-cookbooks repository provides production-ready implementations for using Claude with Vision to interpret charts, graphs, and slide decks. By treating PDFs as multimodal documents, Claude processes both OCR-derived text and visual layout simultaneously, enabling natural language queries about bar heights, trend lines, and color-coded data series.

Prerequisites and Model Requirements

Only claude-sonnet-4-6 supports PDF vision capabilities. Attempting to use other models will result in a validation error. While the feature remains in beta, you must include the header anthropic-beta: pdfs-2024-09-25 on every request.

Install the Anthropic Python SDK and import the required modules:

import base64
from anthropic import Anthropic

Encoding PDFs for Claude Vision

The first step is converting your PDF to a Base64 string. In multimodal/reading_charts_graphs_powerpoints.ipynb, lines 94-98 demonstrate reading the binary file and encoding it for the API payload:

# Read and encode the PDF

with open("./documents/cvna_2021_annual_report.pdf", "rb") as f:
    pdf_bytes = f.read()
base64_pdf = base64.b64encode(pdf_bytes).decode("utf-8")

This encoded string becomes the data source for Claude's document vision processing.

Building the Document Payload

Unlike standard image inputs, chart and graph interpretation uses a document content type with media_type: "application/pdf". According to the implementation in multimodal/reading_charts_graphs_powerpoints.ipynb (lines 30-45), construct your message with a document block followed by your text prompt:

messages = [{
    "role": "user",
    "content": [
        {
            "type": "document",
            "source": {
                "type": "base64",
                "media_type": "application/pdf",
                "data": base64_pdf,
            },
        },
        {"type": "text", "text": "What are the key revenue trends shown in this chart?"},
    ],
}]

Calling the Claude API with Beta Headers

Initialize the client with the required beta header, then wrap the API call in a helper function. Lines 59-66 and 77-82 in multimodal/reading_charts_graphs_powerpoints.ipynb show the complete setup:

# Initialize with PDF beta header

client = Anthropic(default_headers={"anthropic-beta": "pdfs-2024-09-25"})
MODEL_NAME = "claude-sonnet-4-6"

# Helper function from the cookbook

def get_completion(messages):
    resp = client.messages.create(
        model=MODEL_NAME,
        max_tokens=8192,
        temperature=0,
        messages=messages,
    )
    return resp.content[0].text

# Execute the analysis

print(get_completion(messages))

The get_completion helper standardizes token limits and temperature settings while extracting the plain text response from Claude's output.

Processing Multiple Questions on the Same Document

You can query the same PDF multiple times without re-encoding. The cookbook demonstrates this pattern in lines 94-118 of multimodal/reading_charts_graphs_powerpoints.ipynb:

questions = [
    "What was CVNA revenue in 2020?",
    "How many additional markets has Carvana added since 2014?",
    "What was 2016 revenue per retail unit sold?",
]

for q in questions:
    msgs = [{
        "role": "user",
        "content": [
            {"type": "document",
             "source": {"type": "base64",
                        "media_type": "application/pdf",
                        "data": base64_pdf}},
            {"type": "text", "text": q},
        ],
    }]
    print(f"\n--- {q} ---")
    print(get_completion(msgs))

Extracting Structured Narrations from Slide Decks

For multi-page slide decks, prompt Claude to return structured XML tags that you can parse programmatically. As shown in lines 30-38 and 86-92 of the cookbook, request narration wrapped in specific tags, then extract them with regex:

prompt = """You are the Twilio CFO narrating the Q4 2023 earnings deck.
Please describe every visual element on each slide, preserving numbers and colors.
Return the result wrapped in <narration>...</narration> tags."""

messages = [{
    "role": "user",
    "content": [
        {"type": "document",
         "source": {"type": "base64",
                    "media_type": "application/pdf",
                    "data": base64_pdf}},
        {"type": "text", "text": prompt},
    ],
}]
narration_raw = get_completion(messages)

# Extract the XML block (lines 86-92)

import re
match = re.search(r"<narration>(.*?)</narration>", narration_raw, re.DOTALL)
narration = match.group(1) if match else "No narration captured"

Summary

• Model requirement: Only claude-sonnet-4-6 supports PDF vision analysis according to the anthropics/claude-cookbooks source code.
• Beta header: You must include anthropic-beta: pdfs-2024-09-25 in your client initialization while the feature is in preview.
• Encoding workflow: Convert PDFs to Base64 strings and wrap them in a document type content block with media_type: "application/pdf".
• Page limits: A single request supports up to 100 PDF pages; split larger documents and stitch responses together.
• Accuracy tips: For color-sensitive charts, ask Claude to list HEX codes explicitly, and consider adding calculator tools for precise arithmetic on large numbers extracted from charts.

Frequently Asked Questions

Which Claude model supports PDF vision for chart analysis?

Only claude-sonnet-4-6 currently supports PDF vision capabilities. Using any other model will return a validation error. Check the multimodal/reading_charts_graphs_powerpoints.ipynb file in the anthropic/claude-cookbooks repository for the official model specifications.

Can I analyze individual chart images instead of PDFs?

Yes. For single-image charts, use the generic image-vision payload demonstrated in multimodal/getting_started_with_vision.ipynb (lines 78-98). The PDF method is specifically optimized for multi-page documents containing multiple charts and graphs.

What is the page limit for PDF analysis?

A single API request can process up to 100 PDF pages. If your document exceeds this limit, you must split the PDF into chunks, process each separately, and combine the results programmatically.

How accurate is Claude at reading numerical data from charts?

Claude accurately identifies axis labels, bar heights, and data points, but its internal calculator can occasionally slip on large numbers or complex calculations. For financial data requiring precise arithmetic, implement a calculator tool via Claude's tool use feature or post-process the extracted numbers with Python's arithmetic functions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}