How to Extract Form Field Information from PDFs: A Complete Guide to Form Discovery

The extract_form_field_info.py script in the ComposioHQ/awesome-claude-skills repository provides a complete pipeline for discovering interactive PDF form fields and exporting their metadata as structured JSON using the pypdf library.

The ComposioHQ/awesome-claude-skills project includes robust utilities for PDF document processing. When you need to extract form field information from PDFs programmatically, the implementation in document-skills/pdf/scripts/extract_form_field_info.py offers a production-ready solution that handles complex field hierarchies, radio button groups, and coordinate mapping.

Understanding the PDF Form Extraction Pipeline

The extraction process operates in distinct phases to transform raw PDF objects into clean, actionable metadata. The script separates field discovery, type conversion, annotation mapping, and output generation into discrete functions for maintainability.

At the core, the PdfReader class from pypdf opens the document and provides access to the internal form dictionary. Unlike simple text extraction, this approach inspects the PDF's interactive form architecture, including the AcroForm dictionary and page annotation arrays.

Step-by-Step Field Discovery Process

Reading Raw Field Definitions

The entry point uses PdfReader.get_fields() to obtain the complete dictionary of form field objects defined in the PDF. This method returns the raw PDF object tree, including container fields that may hold child elements.

from pypdf import PdfReader

reader = PdfReader("input.pdf")
raw_fields = reader.get_fields()

Filtering Container Fields and Leaf Nodes

PDF forms often group fields using parent-child relationships via the /Kids entry. The script filters these container fields, keeping only leaf nodes while maintaining awareness of radio-button groups. This ensures each extractable field corresponds to a single interactive element on the page.

Normalizing Field IDs via Parent Chain

For annotations, the helper function get_full_annotation_field_id walks up the annotation tree using the /Parent entry to construct dotted field names (e.g., address.street). This mirrors the identifier format used by PdfReader.update_page_form_field_values, ensuring compatibility with subsequent fill operations.

Working with Field Types and Value Constraints

The make_field_dict function translates the PDF field type (/FT) into human-readable descriptors. The script handles four primary field categories:

Text Fields – Extracted as type text with no additional constraints beyond standard PDF field flags.

Checkbox Fields – The script extracts the "checked" and "unchecked" values from the /States_ array (often /Yes and /Off). This captures the specific naming conventions used by the PDF creator, not just Boolean states.

Choice Fields – For dropdown menus and list boxes, the script builds an array of {value, text} pairs for each available option, preserving both the internal PDF value and the display text.

Unknown Types – Any unrecognized /FT value is reported as unknown with the original type preserved for debugging.

# Sample field descriptor output

{
  "field_id": "preferred_contact",
  "type": "choice",
  "choice_options": [
    {"value": "/Email", "text": "Email"},
    {"value": "/Phone", "text": "Phone"}
  ],
  "page": 1,
  "rect": [100, 500, 200, 520]
}

Mapping Fields to Page Annotations and Coordinates

Interactive PDF fields exist as annotations on specific pages. The script iterates over every page’s /Annots array to link field definitions to their visual representations:

1. Obtains the full field ID for each annotation
2. Attaches the page number (1-based indexing)
3. Records the bounding rectangle (/Rect) in PDF coordinates

For radio-button groups, when an annotation belongs to a known group name, the script extracts the active value from the /AP (appearance) dictionary and aggregates each option with its specific rectangle under a single "radio_group" entry.

Fields lacking corresponding annotations are pruned from the output with diagnostic console messages, ensuring the JSON contains only fields that can actually be located within the document.

Sorting for Deterministic Output

To ensure consistent results across runs, fields are sorted first by page number, then by Y-position (top-to-bottom in PDF coordinates) and X-position. This creates a logical reading order that matches the visual layout of the form.

Usage Examples

Command-Line Execution

Process a PDF and export metadata to JSON:

python document-skills/pdf/scripts/extract_form_field_info.py input.pdf fields.json

Programmatic Integration

Import the functions directly into your Python workflow:

from pypdf import PdfReader
from document-skills.pdf.scripts.extract_form_field_info import get_field_info, write_field_info

# Load PDF and obtain field metadata

reader = PdfReader("contract.pdf")
field_info = get_field_info(reader)

# Access field coordinates and types directly

for field in field_info:
    print(f"{field['field_id']}: {field['type']} on page {field['page']}")

# Save to structured JSON

with open("extracted_fields.json", "w") as f:
    json.dump(field_info, f, indent=2)

Sample JSON Output Structure

[
  {
    "field_id": "applicant.name",
    "type": "text",
    "page": 1,
    "rect": [100, 700, 300, 720]
  },
  {
    "field_id": "applicant.newsletter",
    "type": "checkbox",
    "checked_value": "/Yes",
    "unchecked_value": "/Off",
    "page": 1,
    "rect": [100, 650, 120, 670]
  },
  {
    "field_id": "payment.method",
    "type": "radio_group",
    "options": [
      {"value": "/Card", "page": 2, "rect": [100, 500, 120, 520]},
      {"value": "/Cash", "page": 2, "rect": [150, 500, 170, 520]}
    ]
  }
]

Integration with Form Filling

The extracted metadata serves as the input for the companion script document-skills/pdf/scripts/fill_fillable_fields.py. By using the same field_id format and coordinate system, these utilities create a closed loop: extract field definitions, populate values programmatically, and generate completed PDFs.

Summary

• extract_form_field_info.py provides complete PDF form discovery using pypdf.PdfReader and the get_fields() method
• Field hierarchy resolution uses /Parent chain walking to create dotted field identifiers compatible with update_page_form_field_values
• Type detection handles text, checkbox, choice, and radio-group fields, extracting specific values like /States_ for checkboxes and /AP for radio options
• Coordinate mapping links every field to its page number and bounding rectangle via the /Annots array
• Deterministic output sorts fields by page and position to ensure consistent JSON generation
• Companion tooling includes fill_fillable_fields.py for completing forms using the extracted metadata

Frequently Asked Questions

What Python library does the script use to read PDF files?

The script uses pypdf (specifically PdfReader) to open documents and access form fields. This library provides the get_fields() method for extracting raw field objects and update_page_form_field_values for later population, making it well-suited for interactive form workflows.

How does the script handle nested or grouped form fields?

The script resolves nested fields by walking the /Parent chain via get_full_annotation_field_id, constructing dotted field names like address.street or employee.0.name. For radio-button groups, it aggregates individual annotations under a single radio_group entry while preserving each option's specific bounding rectangle.

Can the script extract the exact coordinates of form fields?

Yes. By iterating through each page's /Annots array, the script attaches the /Rect array (containing four coordinates defining the bounding box) to every field descriptor. These coordinates use the PDF coordinate system (origin at bottom-left) and enable precise overlay or clicking automation.

What happens if a form field definition exists but has no visible annotation?

Fields lacking corresponding annotations in the page /Annots arrays are filtered out during the pruning phase. The script prints diagnostic messages to the console identifying these orphan fields, ensuring the final JSON contains only fields that can be physically located and interacted with in the document.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}