# How to Use `--pre` and `--preproc` Flags for Custom Input Filtering in ripgrep

> Learn to use ripgrep's --pre flag for custom input filtering. Execute external preprocessors on files before searching for powerful text manipulation.

- Repository: [Andrew Gallant/ripgrep](https://github.com/BurntSushi/ripgrep)
- Tags: how-to-guide
- Published: 2026-03-05

---

**Use the `--pre <COMMAND>` flag to execute an external preprocessor on files before searching, passing the file path as the first argument and raw content via stdin, while the legacy `--preproc` alias remains functional but undocumented.**

The BurntSushi/ripgrep repository enables powerful text extraction from binary or complex formats through its custom input filtering interface. By intercepting file content before pattern matching, you can search compressed archives, PDFs, and proprietary formats as if they were plain text.

## Internal Architecture of the Preprocessor System

The preprocessor configuration resides in the `SearchConfig` struct defined in [`crates/core/search.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/core/search.rs) (lines 20-22), which stores the command path in the `preprocessor` field and glob patterns in `preprocessor_globs`. When parsing arguments, `--pre` and `--pre-glob` are defined in [`crates/core/flags/defs.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/core/flags/defs.rs) (lines 5488-5500), while [`crates/core/flags/hiargs.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/core/flags/hiargs.rs) wires these values into the search configuration.

Before opening each file, ripgrep invokes the `should_preprocess` function (lines 284-291) to validate that a preprocessor exists and that the target file matches any specified `--pre-glob` constraints. Upon validation, `search_preprocessor` (lines 294-317) manages the external process lifecycle.

## Command Execution and Data Flow

The subprocess protocol follows a strict stdin/stdout contract. RiFgrep spawns the preprocessor with the **target file path as the first positional argument** and pipes the **raw file bytes to stdin**. The external command must write transformed text to stdout, which ripgrep then searches as if it were the original file content.

When preprocessors fail, `search_preprocessor` surfaces explicit errors using this format:

```text
preprocessor command failed: '"./preprocess" "file.pdf"': <error>

```

This immediate feedback simplifies debugging of custom filtering scripts.

## Practical Examples

### Extracting Text from PDF Files

Convert binary documents to searchable text using standard Unix tools. Create an executable preprocessor script:

```bash
#!/bin/sh
exec pdftotext - -

```

Execute a search against PDF content:

```bash
rg --pre ./preprocess 'The Commentz-Walter algorithm' 1995-watson.pdf

```

### Building Conditional Preprocessor Scripts

Handle mixed repositories by filtering specific extensions while passing others through unchanged:

```bash
cat > preprocessor <<'EOF'
#!/bin/sh
case "$1" in
  *.pdf)
    if [ -s "$1" ]; then
      exec pdftotext - -
    else
      exec cat
    fi
    ;;
  *)
    exec cat
    ;;
esac
EOF
chmod +x preprocessor

```

Run the conditional filter across your codebase:

```bash
rg --pre ./preprocessor 'fn is_empty' -c

```

### Restricting Preprocessing with `--pre-glob`

Minimize performance overhead by limiting preprocessor invocation to specific patterns. The `should_preprocess` function evaluates these globs before spawning external processes:

```bash
rg --pre ./preprocessor --pre-glob '*.pdf' 'search term' .

```

Only files matching `*.pdf` trigger the external script; all other files proceed directly to the search phase.

### Using the Legacy `--preproc` Syntax

Older releases accepted `--preproc` as the primary flag. While current documentation favors `--pre`, the alias persists in [`crates/core/flags/defs.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/core/flags/defs.rs) for backward compatibility:

```bash
rg --preproc ./preprocess 'pattern' file.pdf

```

Functionally equivalent to `--pre`, this syntax may emit deprecation warnings in future releases.

### Chaining with Other Flags

Combine preprocessing with standard search modifiers like case-insensitive matching:

```bash
rg -i --pre ./preprocess --pre-glob '*.pdf' 'error' .

```

The preprocessor transforms the PDF content before ripgrep applies the `-i` flag to the resulting text stream.

## Summary

- The **`--pre` flag** stores the external command path in `SearchConfig.preprocessor` within [`crates/core/search.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/core/search.rs), enabling dynamic content transformation.
- **`--pre-glob`** leverages the `preprocessor_globs` field to restrict filtering to specific file patterns, evaluated by the `should_preprocess` function.
- The preprocessor receives the **file path as the first argument** and **raw bytes via stdin**, and must output searchable text to stdout.
- **`--preproc`** serves as a deprecated alias for `--pre`, maintained in [`crates/core/flags/defs.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/core/flags/defs.rs) for legacy script compatibility.
- Execution errors trigger explicit messages from `search_preprocessor`, providing clear diagnostics for debugging custom input filtering pipelines.

## Frequently Asked Questions

### What is the difference between `--pre` and `--preproc`?

There is no functional difference; `--preproc` is the deprecated predecessor to `--pre`. According to the flag definitions in [`crates/core/flags/defs.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/core/flags/defs.rs), both populate the identical internal configuration field. Modern releases document only `--pre`, though `--preproc` remains parseable for backward compatibility.

### How does ripgrep pass data to the preprocessor?

As implemented in the `search_preprocessor` function in [`crates/core/search.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/core/search.rs) (lines 294-317), ripgrep spawns the external command with the target file path as the first positional argument and pipes the raw file contents directly to the subprocess's stdin. The preprocessor must write the transformed text to stdout for ripgrep to consume.

### Can I use `--pre` with binary files like PDFs or DOCX?

Yes, converting binary formats to text represents the primary use case for custom input filtering. Your preprocessor script can invoke utilities like `pdftotext`, `antiword`, or custom parsers to extract searchable content before ripgrep executes pattern matching against the transformed stream.

### How do I limit which files trigger the preprocessing overhead?

Use the `--pre-glob` flag to define inclusion patterns such as `*.pdf` or `*.docx`. The `should_preprocess` function (lines 284-291) checks these globs before invoking your command, ensuring that only matching files incur the performance cost of external process execution.