Detecting Vector Grids Inside PDF Regions for TSR-Compatible Processing with pdf-inspector

pdf-inspector provides the detect_vector_grid_in_region_mem function to detect table structures from vector graphics in arbitrary PDF regions, returning TSR-compatible tokens and cell bounding boxes without file I/O overhead.

Detecting vector grids inside PDF regions for TSR-compatible processing is essential for hybrid OCR pipelines that need precise table geometry without expensive full-page recognition. The pdf-inspector Rust library from Firecrawl solves this with specialized region-scoped APIs that operate entirely in memory. This guide explains how vector grid detection works, how to integrate it into Table Structure Recovery (TSR) workflows, and how the modular architecture enables consistent, high-performance extraction.

How Vector Grid Detection Works in pdf-inspector

The detect_vector_grid_in_region_mem function in src/lib.rs (lines ~12001–13000) provides a complete in-memory pipeline for identifying table structures from PDF vector graphics. It follows a seven-step process that reuses the same geometry extraction primitives as full-document processing.

Step-by-Step Detection Flow

1. Load PDF and resolve page: Calls load_document_from_mem once, then obtains the target page ID for the requested page index.

2. Fast font CMap extraction: Uses FontCMaps::from_doc_pages_fast scoped to the required page—avoiding expensive TrueType fallbacks while preserving ToUnicode handling.

3. Extract raw geometry: Invokes extractor::content_stream::extract_page_text_items to obtain three critical collections:

   • Vec<TextItem> — positioned text with font metadata
   • Vec<PdfRect> — filled rectangles (potential cell backgrounds)
   • Vec<PdfLine> — stroke paths (ruling lines)

4. Coordinate transformation: Converts the caller's PDF-point region into the extractor's coordinate space via region_bounds. Exits early if the page is rotated, as TSR requires standard orientation.

5. Region filtering: Builds items_in_region, rects_in_region, and lines_in_region using overlap tests against the transformed bounds.

6. Table detector cascade: Runs two strategies in priority order:

   • Rect-backed detection: tables::detect_tables_from_rects identifies grids from rectangular cell boundaries
   • Line-backed detection: tables::detect_vector_grid_tables_from_lines constructs tables from ruling lines

7. Result conversion: vector_grid_result_from_table transforms successful detections into VectorGridDetection with structure_tokens (e.g., <td>) and pixel-space cell_bboxes.

The function returns Some(VectorGridDetection) on first success or None if no grid is detected.

Using detect_vector_grid_in_region_mem in Your Code

The API accepts raw PDF bytes, a page index, region bounds in PDF points, and the rendering DPI used by your caller. This design eliminates file I/O and integrates cleanly with layout models that provide bounding-box predictions.

Single Region Detection

use pdf_inspector;

fn detect_table_grid(pdf_bytes: &[u8]) -> Result<(), Box<dyn std::error::Error>> {
    let page_index = 0;                         // 0-indexed page number
    let region = [50.0, 700.0, 550.0, 100.0];   // PDF points: (x1, y1, x2, y2)
    let dpi = 300.0;                             // Must match your renderer's DPI

    match pdf_inspector::detect_vector_grid_in_region_mem(
        pdf_bytes,
        page_index,
        region,
        dpi,
    )? {
        Some(grid) => {
            // TSR-compatible outputs
            println!("Structure tokens: {:?}", grid.structure_tokens);
            // [["<td>", "<td>"], ["<td>", "<td>"]] for 2x2 table
            
            for (i, bbox) in grid.cell_bboxes.iter().enumerate() {
                // Pixel-space coordinates: [x0, y0, x1, y1]
                println!("Cell {}: {:?}", i, bbox);
            }
        }
        None => println!("No vector grid detected—consider OCR fallback"),
    }
    
    Ok(())
}

Batch Region Extraction with OCR Routing

For multiple regions across multiple pages, use extract_text_in_regions_mem to get both native text and OCR-needs flags:

use pdf_inspector;

fn process_regions(pdf_bytes: &[u8]) -> Result<(), Box<dyn std::error::Error>> {
    // Define regions as (page_index, Vec<[x1, y1, x2, y2]>)
    let regions = [
        (0, vec![
            [50.0, 700.0, 550.0, 100.0],   // Header table
            [60.0, 600.0, 540.0, 200.0],   // Body table
        ]),
        (2, vec![
            [30.0, 750.0, 560.0, 120.0],   // Page 3 summary table
        ]),
    ];

    let results = pdf_inspector::extract_text_in_regions_mem(pdf_bytes, &regions)?;
    
    for page in results {
        println!("Page {}", page.page);
        
        for (i, region) in page.regions.iter().enumerate() {
            if region.needs_ocr {
                // Route to OCR pipeline—likely scanned content or complex encoding
                println!("  Region {}: needs OCR", i);
            } else {
                // Native extraction succeeded
                println!("  Region {}: {} chars", i, region.text.len());
            }
        }
    }
    
    Ok(())
}

The Table Detection Architecture

The tables module (src/tables/mod.rs) implements three complementary strategies that share geometry primitives with region-scoped extraction.

Rect-Backed Detection (src/tables/detect_rects.rs)

Analyzes PdfRect collections to identify tables by their filled cell backgrounds. Effective for:

• Grid cells with colored backgrounds
• Bounded regions without visible ruling lines
• PDFs where rectangles define cell extents more reliably than strokes

Line-Backed Detection (src/tables/detect_lines.rs)

Processes PdfLine vectors to reconstruct tables from ruling lines. Handles:

• Traditional ruled tables with visible borders
• Partial grids where lines define only some boundaries
• Complex nested structures by line intersection analysis

Heuristic Fallback (src/tables/detect_heuristic.rs)

Operates on TextItem positioning alone when no clear geometric grid exists. Used for:

• Whitespace-aligned tables without graphics
• Irregular or ragged-right column structures

All three paths converge on vector_grid_result_from_table to produce TSR-compatible output, ensuring consistent token formatting regardless of detection method.

Integration with Hybrid OCR Pipelines

pdf-inspector's modular design enables sophisticated workflows where layout models supply regions and the library supplies structure:

Full Pipeline with Automatic Routing

use pdf_inspector::{PdfOptions, ProcessMode};

fn full_pipeline(path: &str) -> Result<(), Box<dyn std::error::Error>> {
    let opts = PdfOptions::new()
        .mode(ProcessMode::Full);
    
    let result = pdf_inspector::process_pdf_with_options(path, opts)?;
    
    println!("PDF type: {:?}", result.pdf_type);
    // Native | Scanned | Mixed
    
    if let Some(md) = result.markdown {
        println!("Clean markdown: {} bytes", md.len());
    }
    
    // Pages requiring OCR—route these to Tesseract/Cloud Vision
    println!("OCR needed on pages: {:?}", result.pages_needing_ocr);
    
    Ok(())
}

Layout Complexity Signals

The detector module (src/detector.rs) provides OCR routing signals used throughout the pipeline:

Signal	Trigger	Constant
OCR_REASON_VECTOR_TEXT	Vector-outlined text detected	src/lib.rs lines 17–19
Large raster images	Scanned page content	Image byte threshold
GID-encoded fonts	Unmapped glyph IDs	Font analysis
Encoding failures	ToUnicode mapping gaps	src/tounicode.rs

These signals drive decisions in compute_layout_complexity_with_chart_regions, ensuring OCR is invoked only when native extraction is unreliable.

Performance Characteristics

• Zero file I/O: detect_vector_grid_in_region_mem operates on byte slices—ideal for serverless or memory-constrained environments
• Single PDF load: Document parsing happens once regardless of region count
• Fast font handling: Page-scoped FontCMaps::from_doc_pages_fast avoids global TrueType fallback
• Early exits: Rotated pages fail immediately; empty regions short-circuit detection

The shared geometry pipeline between extract_pages_markdown_mem (full document) and region-scoped helpers guarantees consistent detection behavior across usage patterns.

Key Source Files

File	Purpose
src/lib.rs	Public API, detect_vector_grid_in_region_mem, PdfOptions builder
src/detector.rs	PDF classification, OCR routing signals
src/extractor/mod.rs	Content stream parsing, extract_page_text_items
src/tables/mod.rs	Table detection orchestration, vector_grid_result_from_table
src/tables/detect_rects.rs	Rect-backed grid detection
src/tables/detect_lines.rs	Line-backed vector grid detection
src/tables/detect_heuristic.rs	Text-position heuristic fallback
src/tounicode.rs	ToUnicode map parsing for glyph accuracy

Summary

• In-memory operation: detect_vector_grid_in_region_mem requires no temporary files—pass raw PDF bytes and receive TSR-compatible grid descriptions
• Dual detection strategy: Rect-backed and line-backed detectors cover diverse PDF table constructions, with heuristic fallback for edge cases
• Consistent geometry reuse: The same extraction primitives serve full-document and region-scoped APIs, eliminating detection drift
• OCR-aware design: Built-in signals like OCR_REASON_VECTOR_TEXT enable intelligent hybrid pipelines that minimize expensive recognition passes
• Rust-native performance: Zero-copy parsing and scoped font handling make the library suitable for high-throughput document processing services

Frequently Asked Questions

What does TSR-compatible output mean?

TSR-compatible output follows conventions expected by Table Structure Recovery pipelines: a sequence of structure tokens (e.g., <td>, </tr>) paired with pixel-space bounding boxes for each cell. This format allows downstream models to associate visual regions with semantic table structure without re-deriving geometry from raw OCR text.

When should I use region-scoped detection versus full-document extraction?

Use detect_vector_grid_in_region_mem when you have pre-identified table regions from a layout model and need precise cell geometry. Use full-document extraction (process_pdf_with_options) when processing complete documents without prior region knowledge. The region API avoids work on non-table areas and enables targeted OCR fallback.

Why does the function reject rotated pages?

TSR pipelines typically assume standard page orientation for coordinate transformations and cell indexing. The function checks rotation at lines 12031–12038 in src/lib.rs and returns None for non-zero rotation angles. Pre-rotate your PDF or use a layout model that supplies de-rotated regions.

How do I handle GID-encoded fonts that lack ToUnicode maps?

The detector flags these via encoding analysis in src/tounicode.rs. Such regions will show needs_ocr: true in extract_text_in_regions_mem results. Route these to your OCR pipeline—pdf-inspector intentionally avoids guessing glyph meanings when mapping is unavailable.

Can I use this without the Rust toolchain?

pdf-inspector provides CLI tools built from the same library. For non-Rust environments, wrap the library with wasm-bindgen for JavaScript/TypeScript, or use gRPC/HTTP bindings if you deploy the CLI as a microservice.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}