# How PDF-Inspector Avoids Redundant I/O Between Detection and Extraction Stages

> Learn how PDF-Inspector prevents redundant I/O by using a single in-memory lopdf::Document shared between detection and extraction for efficient PDF processing.

- Repository: [Firecrawl/pdf-inspector](https://github.com/firecrawl/pdf-inspector)
- Tags: internals
- Published: 2026-08-14

---

**PDF-Inspector eliminates duplicate file reads by loading a PDF into a single in-memory `lopdf::Document` that is shared between both the detection and extraction pipelines.**

The `firecrawl/pdf-inspector` Rust library processes PDFs through two distinct stages—first detecting document properties (type, page count, OCR requirements), then extracting structured text. Rather than parsing the file twice, the library constructs one `Document` instance and passes it to both stages. This design choice significantly reduces disk I/O and CPU overhead, especially for large PDFs or high-throughput applications.

## The Single-Load Architecture

At the core of this optimization sits the `process_pdf_with_options` function in [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs). This public API entry point orchestrates the entire pipeline with a single file read.

```rust
/// Process a PDF file with custom options.
///
/// The document is loaded **once** and shared between detection and extraction.
pub fn process_pdf_with_options<P: AsRef<Path>>(
    path: P,
    options: PdfOptions,
) -> Result<PdfProcessResult, PdfError> {
    let start = ProcessingTimer::start();
    validate_pdf_file(&path)?;

    // Load the document once — shared by detection **AND** extraction.
    let (doc, page_count) =
        load_document_from_path_with_password(&path, options.password.as_deref())?;

    process_document(doc, page_count, options, start)
}

```

The `(doc, page_count)` tuple returned here becomes the foundation for all subsequent operations. Notice that `doc` is an owned `lopdf::Document`, not a path or buffer that would require re-parsing.

## How the Document Is Loaded

The loading logic resides in `load_document_from_path_with_password`, which performs exactly one filesystem read:

```rust
/// Load a PDF file, decrypting with `password` if the file is encrypted.
pub(crate) fn load_document_from_path_with_password<P: AsRef<Path>>(
    path: P,
    password: Option<&str>,
) -> Result<(Document, u32), PdfError> {
    let buffer = std::fs::read(&path)?;
    load_document_from_mem_with_password(&buffer, password)
}

```

This function reads the entire file into a `Vec<u8>` buffer, then delegates to `load_document_from_mem_with_password` to handle decryption and PDF structure parsing. The resulting `Document` contains the fully parsed PDF cross-reference table, object tree, and page hierarchy—expensive data that would be wasteful to reconstruct.

## Detection Stage: Zero Additional I/O

After loading, `process_document` immediately hands the `doc` reference to the detection pipeline:

```rust
let detection = detector::detect_from_document(&doc, page_count, &options.detection)?;

```

The `detect_from_document` function in [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs) inspects the already-loaded structure to determine:

- PDF version and compliance level
- Presence of scanned pages requiring OCR
- Font embedding and encoding schemes
- Security settings and encryption status

Because the `Document` is passed by reference (`&doc`), the detection stage performs **no file operations whatsoever**. It traverses the in-memory object graph that `lopdf` has already built.

## Extraction Stage: Reusing the Same Document

Following detection, extraction functions in [`src/extractor/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/mod.rs) and its submodules receive the same `&doc` reference. For example, `extractor::extract_positioned_text_from_doc` operates directly on this shared document to:

- Decode content streams
- Map character codes to Unicode
- Reconstruct reading order and layout

Since the PDF's cross-reference table and stream objects remain accessible in memory, extraction proceeds without re-reading the source file or re-parsing the PDF's structural metadata.

## Memory-Buffer Variants for Network Workflows

The same single-load pattern extends to in-memory APIs for scenarios where PDFs arrive from HTTP requests, databases, or message queues:

```rust
let bytes = std::fs::read("sample.pdf").unwrap();
let result = pdf_inspector::process_pdf_mem(&bytes).unwrap(); // one parse

```

Under the hood, `process_pdf_mem` calls `load_document_from_mem_with_password`, which bypasses `std::fs::read` but otherwise follows the identical path: parse once, share everywhere. Both `detect_pdf_mem` and the full `process_pdf_mem` pipeline use this same entry point.

## Practical Usage Patterns

### Detection-Only Mode (Fast Metadata)

When you need only document properties without text extraction, use the detect-only pathway to minimize work:

```rust
use pdf_inspector::{detect_pdf, PdfOptions, ProcessMode};

let result = detect_pdf("sample.pdf").unwrap(); // loads once, runs detection
println!("type: {:?}, pages: {}", result.pdf_type, result.page_count);

```

Even in this minimal mode, the document is fully parsed—there is no "lazy" or "partial" loading. However, the extraction stage is skipped entirely, saving the CPU cost of text decoding and layout analysis.

### Full Detect-Then-Extract Pipeline

For complete processing, the default `process_pdf` function handles both stages with guaranteed single I/O:

```rust
use pdf_inspector::{process_pdf, PdfOptions};

let result = process_pdf("sample.pdf").unwrap(); // one load, shared doc
println!("{}", result.markdown.unwrap());

```

The `PdfProcessResult` contains both the `detection` metadata and the final `markdown` string, all derived from one `lopdf::Document` instance.

## Key Implementation Files

| File | Responsibility |
|------|--------------|
| [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) | Public API entry points, single-load orchestration in `process_pdf_with_options` |
| [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs) | `detect_from_document`—consumes shared `&Document` for metadata extraction |
| [`src/extractor/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/mod.rs) | Text extraction utilities that accept the same `&Document` reference |
| [`src/extractor/content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/content_stream.rs) | Low-level content stream parsing using the loaded document's object cache |
| [`src/extractor/layout.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/layout.rs) | Layout analysis operating on items extracted from the shared document |

## Summary

- **One `fs::read` per file**: The `load_document_from_path_with_password` function in [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) performs the sole filesystem read.
- **Shared `lopdf::Document`**: Both `detector::detect_from_document` and extractor functions receive the same in-memory document reference.
- **No re-parsing**: The PDF's cross-reference table, object tree, and page hierarchy are built exactly once.
- **Symmetric APIs**: File-based and memory-buffer variants (`process_pdf`/`process_pdf_mem`) follow identical single-load semantics.
- **Performance benefit**: Eliminates redundant I/O and CPU-intensive PDF parsing, especially critical for batch processing or large documents.

## Frequently Asked Questions

### How does PDF-Inspector handle password-protected PDFs without reading the file twice?

The password is passed through `load_document_from_path_with_password` and applied during the initial parse by `load_document_from_mem_with_password`. The decrypted `Document` is then shared with both detection and extraction stages. No second decryption or file read occurs.

### Can I use detection results to decide whether to run extraction?

Yes. The `process_pdf_with_options` function accepts a `PdfOptions` struct that includes `ProcessMode`. You can inspect detection results from a preliminary call and conditionally invoke full extraction, or use `PdfOptions` to configure extraction behavior based on detected PDF type—all without reloading the document if you structure your own wrapper to cache the `Document`.

### What happens if the PDF is corrupted or malformed?

`load_document_from_path_with_password` returns a `PdfError` during the initial parse if `lopdf` cannot interpret the file structure. Both detection and extraction are aborted since they depend on the successfully loaded `Document`. This fail-fast behavior prevents wasted work on unreadable inputs.

### Is the same optimization available for streaming or chunked PDF sources?

The current implementation requires the complete file in memory (via `std::fs::read` or provided buffer) because `lopdf` needs random access to the PDF's cross-reference table. Streaming sources must be fully buffered before parsing; the single-load optimization then applies to the resulting in-memory document.