How Claude Handles PDF and Image Inputs When Making API Calls

Claude accepts both PDF and image data as Base64-encoded binary blobs within the messages payload, treating PDFs as type: "document" and images as type: "image" in the Anthropic API request structure.

The asgeirtj/system_prompts_leaks repository reveals the internal system prompts that govern how Claude processes multimodal inputs. According to the source code analysis, Claude handles PDF and image inputs by converting them to Base64-encoded binary assets and embedding them directly into the API request's messages array, with specific handling rules to optimize context window usage.

File Type Detection and Classification

When a user uploads a file, Claude first determines whether the file is plain text (e.g., .md, .txt) or a binary asset. According to the system message rules in Anthropic/claude-opus-4.6.md (lines 77-84), PDFs and PNG/JPG files are classified as image-type assets and are kept as Base64 blobs. The prompt explicitly notes that PDFs are treated "as image" within the UI and API context.

Base64 Encoding Process

For binary files that are not already present in the context window, Claude uses a local FileReader to read the raw bytes. The handling notes in Anthropic/old/claude-opus-4.5.md (lines 75-84) demonstrate the conversion of a PDF file to a Base64 string using the browser FileReader API. The process involves calling readAsDataURL(), then stripping the data:<mime>;base64, prefix to retain only the Base64 data, ensuring the payload fits the JSON schema required by Anthropic.

API Payload Structure

The Base64 payload is embedded in the messages array as a document (for PDFs) or an image (for raster images). Each entry has a type, a source object that specifies media_type and the Base64 data.

PDF Document Schema

As shown in Anthropic/old/claude-opus-4.5.md (lines 95-101), a PDF is embedded using type: "document":

{
  "type": "document",
  "source": {
    "type": "base64",
    "media_type": "application/pdf",
    "data": "<base64-encoded-pdf-data>"
  }
}

Image Schema

For images, the structure uses type: "image" as documented in Anthropic/old/claude-opus-4.5.md (lines 119-126):

{
  "type": "image",
  "source": {
    "type": "base64",
    "media_type": "image/jpeg",
    "data": "<base64-encoded-image-data>"
  }
}

Handling Rules and Context Optimization

Claude only calls the computer (e.g., the view tool) when it needs to process an uploaded binary file. If the file's text is already present in the conversation context (e.g., a PDF has been OCR-processed server-side), Claude will skip the extra step. This optimization is captured in the user-uploaded-files section of Anthropic/claude-opus-4.6.md (lines 88-93), which instructs Claude to ask for clarification when an image/PDF is present and to avoid unnecessary tool calls if the content is already in the context window.

Implementation Examples

Converting a PDF to Base64 (Client-side)

// Assume `file` is a File object from an <input type="file">
async function pdfToBase64(file) {
  return new Promise((resolve, reject) => {
    const reader = new FileReader();
    reader.onload = () => {
      // reader.result is like "data:application/pdf;base64,AAA..."
      const base64 = reader.result.split(',')[1]; // strip prefix
      resolve(base64);
    };
    reader.onerror = () => reject(new Error('Failed to read PDF'));
    reader.readAsDataURL(file);
  });
}

Source: Conversion steps shown in Anthropic/old/claude-opus-4.5.md (lines 75-84).

Building the Anthropic API Payload (PDF)

const base64Pdf = await pdfToBase64(pdfFile);

const payload = {
  model: "claude-sonnet-4-20250514",
  max_tokens: 1024,
  messages: [
    {
      role: "user",
      content: [
        {
          type: "document",
          source: {
            type: "base64",
            media_type: "application/pdf",
            data: base64Pdf,
          },
        },
        { type: "text", text: "Summarize the main conclusions of this paper." },
      ],
    },
  ],
};

fetch("https://api.anthropic.com/v1/messages", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    // API key is injected by the backend; callers do not need to provide it.
  },
  body: JSON.stringify(payload),
})
  .then((r) => r.json())
  .then((data) => console.log(data.content[0].text));

Source: Payload example in Anthropic/old/claude-opus-4.5.md (lines 95-101).

Building the Anthropic API Payload (Image)

const base64Img = await imageToBase64(jpgFile); // same helper as PDF, but mime = image/jpeg

const payload = {
  model: "claude-opus-4-5-20251101",
  max_tokens: 512,
  messages: [
    {
      role: "user",
      content: [
        {
          type: "image",
          source: {
            type: "base64",
            media_type": "image/jpeg",
            data: base64Img,
          },
        },
        { type: "text", text: "Describe the scene in this picture." },
      ],
    },
  ],
};

Source: Image handling block in Anthropic/old/claude-opus-4.5.md (lines 119-126).

Summary

• Claude processes PDFs and images as Base64-encoded binary blobs embedded directly in the API request's messages array.
• PDFs use type: "document" with media_type: "application/pdf", while images use type: "image" with appropriate MIME types like image/jpeg.
• The system prompts in Anthropic/claude-opus-4.6.md classify PDFs as image-type assets and define specific rules for when Claude should invoke tools to read uploaded files versus using existing context.
• Client-side conversion requires stripping the data:<mime>;base64, prefix from FileReader.readAsDataURL() results to obtain the raw Base64 string required by the Anthropic API schema.

Frequently Asked Questions

What is the maximum file size for PDFs and images sent to Claude?

While the leaked system prompts in the asgeirtj/system_prompts_leaks repository do not specify exact byte limits, the Anthropic API documentation typically enforces limits based on the model context window and individual file size restrictions (often around 32MB per file). The Base64 encoding increases payload size by approximately 33%, so large PDFs may require compression or splitting before encoding to avoid request size errors.

Can Claude process multiple images or PDFs in a single API call?

Yes, the messages array supports multiple content blocks within a single user turn. You can include several type: "image" or type: "document" objects alongside text prompts in the content array, allowing Claude to analyze multiple visual assets simultaneously and cross-reference them in its response. Each binary asset requires its own Base64-encoded source object with the appropriate media_type specified.

Does Claude OCR PDFs automatically or require pre-processing?

According to the system prompts in Anthropic/claude-opus-4.6.md (lines 88-93), Claude treats PDFs as binary image assets and only invokes the computer tool (such as the view tool) when it needs to process the file. If the PDF content has already been OCR-processed server-side and exists in the conversation context, Claude will use that existing text rather than re-processing the binary file, optimizing context window usage and reducing unnecessary tool calls.

What MIME types are supported for image inputs?

Based on the handling notes in Anthropic/old/claude-opus-4.5.md (lines 71-84), Claude explicitly supports image/jpeg and image/png formats, treating them as image-type assets. The API structure allows for other standard image MIME types in the media_type field, but the system prompts specifically enumerate PNG and JPEG as the primary supported formats for binary image uploads.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}