# How ripgrep Detects and Handles Binary Files: LineBuffer and BinaryDetection Explained

> Learn how ripgrep detects binary files using LineBuffer and BinaryDetection. Discover configuration options to manage binary file searching effectively.

- Repository: [Andrew Gallant/ripgrep](https://github.com/BurntSushi/ripgrep)
- Tags: internals
- Published: 2026-03-05

---

**ripgrep detects binary files by scanning input chunks for NUL bytes (0x00) via a configurable `BinaryDetection` policy in the `LineBuffer`, which can either halt searching, convert null bytes to newlines, or disable detection entirely based on the `--binary` or `-a/--text` flags.**

When searching through large codebases, ripgrep (from the BurntSushi/ripgrep repository) must efficiently distinguish between text and binary files to avoid processing non-readable data. The tool implements a sophisticated binary detection system centered around a line buffer that scans for null bytes using a configurable policy. Understanding how ripgrep detects and handles binary files reveals the performance optimizations and safety mechanisms built into this Rust-based search tool.

## Binary Detection Architecture in ripgrep

The core of ripgrep's binary file handling resides in the `LineBuffer` struct located in [`crates/searcher/src/line_buffer.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/searcher/src/line_buffer.rs). As ripgrep reads file contents, it buffers data through this component, which is configured with a **binary detection policy** (`BinaryDetection`) that determines how the searcher treats non-textual data.

### The BinaryDetection Policy Variants

The `BinaryDetection` enum defines three distinct behaviors for handling potential binary content:

- **`None`**: Disables binary detection entirely, passing all data to the matcher unchanged regardless of content.
- **`Quit(u8)`**: Scans every chunk for the specified byte (default `0x00`). When found, the buffer reports EOF at that position, records the absolute offset in `binary_byte_offset`, and the searcher stops processing the file.
- **`Convert(u8)`**: Searches for the same byte but replaces each occurrence with the line terminator (`\n`), allowing the file to be searched as text while preventing raw binary data from reaching the matcher. The first offset of a replaced byte is still reported.

### The LineBuffer Implementation

Within `LineBuffer::fill`, the detection logic checks `self.config.binary` to apply the configured policy. When the `Quit` variant triggers, the buffer sets `binary_byte_offset` and stops further reads. This causes the searcher's `Core::detect_binary` method (in [`crates/searcher/src/searcher/core.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/searcher/src/searcher/core.rs)) to return `true`, subsequently invoking `Sink::binary_data` to emit a warning and suppress additional matches.

## How ripgrep Handles Binary File Detection

Once the `LineBuffer` identifies binary content, the detection flows through the searcher's core architecture. The `Core::detect_binary` method evaluates the buffer's state and coordinates the response based on the active policy.

When binary data is confirmed under the default `Quit` policy, the system invokes `Sink::binary_data`, which generates user-facing warnings such as `WARNING: stopped searching binary file after match (found "\0" byte around offset …)`. This mechanism prevents garbled output while informing users exactly where binary content was encountered. The searcher halts further processing of that file unless configured to convert or ignore binary data.

## Configuring Binary File Handling with CLI Flags

ripgrep exposes three distinct modes for binary file handling through command-line flags defined in [`crates/core/flags/defs.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/core/flags/defs.rs) and parsed in [`crates/core/flags/hiargs.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/core/flags/hiargs.rs). The `BinaryMode` from low-level arguments maps to `BinaryDetection` values via `BinaryDetection::from_low_args`, determining how the searcher processes potential binary content.

**Automatic mode (default)**: Uses `BinaryDetection::Quit(b'\0')`, causing ripgrep to stop searching at the first NUL byte and emit a warning.

**`--binary` flag**: Enables `BinaryDetection::Convert(b'\0')`, which converts null bytes to newlines so the file is treated as searchable text while masking binary characters from output.

**`-a` or `--text` flag**: Sets `BinaryDetection::None`, completely disabling binary detection and processing files exactly as supplied, which may produce garbled terminal output containing raw null characters.

### Practical Examples

```bash

# Default behavior: binary files are skipped with a warning

rg "secret" binary_file.bin

# → WARNING: stopped searching binary file after match (found "\0" byte around offset 12345)

# Search binary files but suppress binary data (replace NUL with newline)

rg --binary "secret" binary_file.bin

# → prints any matches found, binary bytes are not shown

# Treat binary files as pure text (no detection, raw output may be garbled)

rg -a "secret" binary_file.bin

# → prints matches and may include raw binary characters

```

## Key Source Files and Implementation Details

The following files in the BurntSushi/ripgrep repository implement the binary detection system:

- **[`crates/searcher/src/line_buffer.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/searcher/src/line_buffer.rs)**: Implements the `LineBuffer` struct, the `BinaryDetection` enum, and the `fill` logic that scans for the configured byte.
- **[`crates/searcher/src/searcher/core.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/searcher/src/searcher/core.rs)**: Contains `Core::detect_binary` and forwards binary-data events to the sink.
- **[`crates/core/flags/hiargs.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/core/flags/hiargs.rs)**: Translates CLI flags (`--binary`, `-a/--text`) into `BinaryDetection` values used by the searcher.
- **[`crates/core/flags/defs.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/core/flags/defs.rs)**: Documents the `--binary` and `-a/--text` flags, describing their effect on binary handling.
- **[`crates/core/flags/doc/help.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/core/flags/doc/help.rs)**: Generates the user-visible help text that explains binary-file filtering options.

## Summary

- ripgrep uses a `LineBuffer` with a configurable `BinaryDetection` policy to identify binary files by scanning for NUL bytes (0x00) according to the source code in [`crates/searcher/src/line_buffer.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/searcher/src/line_buffer.rs).
- Three detection modes exist: `Quit` (default) stops searching immediately upon finding a null byte, `Convert` replaces null bytes with newlines to enable text searching, and `None` disables detection entirely.
- The detection logic resides in `LineBuffer::fill`, with coordination in `Core::detect_binary` in [`crates/searcher/src/searcher/core.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/searcher/src/searcher/core.rs).
- CLI flags `--binary` and `-a/--text` control the detection policy through the flag parsing logic in [`crates/core/flags/hiargs.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/core/flags/hiargs.rs).
- When binary content is detected under the default policy, `Sink::binary_data` emits warnings and halts processing to prevent corrupt output while reporting the exact offset where binary data was found.

## Frequently Asked Questions

### How does ripgrep determine if a file is binary?

ripgrep scans file contents through a `LineBuffer` looking for null bytes (0x00) by default. When the `BinaryDetection::Quit` policy encounters a null byte, it records the offset in `binary_byte_offset`, reports EOF to stop further reading, and triggers `Core::detect_binary` in [`crates/searcher/src/searcher/core.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/searcher/src/searcher/core.rs) to halt searching and emit a warning message.

### What is the difference between ripgrep's --binary and -a flags?

The `--binary` flag uses `BinaryDetection::Convert(b'\0')`, which replaces null bytes with newlines so the file is searched as text without displaying raw binary characters. The `-a` or `--text` flag sets `BinaryDetection::None`, completely disabling binary detection and passing raw bytes directly to the matcher, which may produce garbled terminal output containing unprintable characters.

### Where in the ripgrep source code is binary detection implemented?

The primary implementation resides in [`crates/searcher/src/line_buffer.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/searcher/src/line_buffer.rs) within the `LineBuffer::fill` method. The `BinaryDetection` enum defines the detection policies, while [`crates/searcher/src/searcher/core.rs`](https://github.com/BurntSushi/ripgrep/blob/main/crates/searcher/src/searcher/core.rs) contains the `Core::detect_binary` method that coordinates the response to binary data detection.

### Can ripgrep search binary files without converting null bytes to newlines?

Yes, using the `-a` or `--text` flag disables binary detection entirely (`BinaryDetection::None`), allowing ripgrep to search binary files without converting null bytes. However, this approach passes raw binary data directly to the output, which may result in garbled terminal displays or broken pipe errors when null bytes or other non-text characters are printed.