How to Search Compressed Files with ripgrep: Gzip, Bzip2, XZ and More

Use the -z or --search-zip flag to transparently search inside gzip, bzip2, xz, lz4, lzma, and zstd compressed files without manually decompressing them first.

The ripgrep command-line tool from the BurntSushi/ripgrep repository treats compressed archives as transparent search targets when you enable compressed file search mode. By default, ripgrep skips compressed files or treats them as binary blobs, but the decompression subsystem allows you to search their contents as if they were plain text streams.

Enable Compressed File Search with the -z Flag

To search compressed files with ripgrep, pass the -z (or --search-zip) flag. This instructs the CLI to move matching files to the decompression subsystem instead of treating them as binary data.

In crates/core/flags/defs.rs (lines 6060-6075), the flag definition enables what the codebase calls "search-zip" mode. When active, ripgrep examines file extensions against a built-in list of compressed types before spawning the appropriate external decompression utility.

rg -z "error_pattern" /var/log/

Without this flag, ripgrep treats .gz, .bz2, and similar files as binary and skips their contents unless you force binary search with --binary, which would match against the compressed byte stream rather than the decompressed content.

Supported Compression Formats and Architecture

ripgrep supports transparent decompression for multiple formats including gzip (.gz), bzip2 (.bz2), xz (.xz), lz4 (.lz4), lzma (.lzma), and zstd (.zst). The architecture relies on external command-line tools rather than internal libraries, streaming decompressed data directly into the search engine.

How ripgrep Detects Compressed Files

The list of recognized compressed file extensions lives in crates/ignore/src/default_types.rs (lines 105-108). When ripgrep encounters a file with a matching extension while the -z flag is active, it flags that file for decompression processing.

The default type system maps extensions to internal type identifiers. For compressed files, these mappings trigger the decompression driver rather than the standard file reader.

The Decompression Pipeline

The core decompression logic resides in crates/cli/src/decompress.rs. This module maintains command tables (around lines 290-298) that map specific extensions to command arrays:

• *.gz → ["gzip", "-d", "-c"]
• *.bz2 → ["bzip2", "-dc"]
• *.xz → ["xz", "-dc"]

When ripgrep identifies a compressed file, it spawns the corresponding external utility via std::process::Command and pipes the command's stdout directly into the core searcher (crates/core/search.rs). The Search struct consumes any Read implementation, allowing it to process the decompressed byte stream without writing temporary files to disk.

Security and Command Execution

The decompression subsystem contains safety checks (documented in crates/cli/src/decompress.rs, lines 409-438) that prevent execution of arbitrary pre-processors. ripgrep only executes the hardcoded decompression commands for known extensions. If you need to search a custom compressed format not natively supported, you must explicitly opt-in using the --pre flag with a custom pre-processor command.

Practical Examples

Search for a pattern inside all gzip and bzip2 files recursively:

rg -z "TODO" .

Search only inside .xz files, limiting directory depth to two levels:

rg -z --type-add 'xz:*.xz' --type xz -g '**/*.xz' -d 2 "FIXME"

Use a custom pre-processor for a format not natively supported (e.g., xzcat for specific xz handling):

rg --pre 'xzcat' -z "critical" logs/

List all compressed files that would be searched without actually searching (dry-run):

rg -z --files-with-matches --no-messages ".*" .

Performance and Streaming Architecture

The decompression implementation emphasizes streaming performance. Rather than extracting archives to temporary files, ripgrep pipes the decompressed output directly into its regex engine. This approach minimizes I/O overhead and memory usage, allowing you to search multi-gigabyte compressed logs as efficiently as searching plain text.

The external tool approach (using system gzip, bzip2, etc.) ensures ripgrep leverages optimized native implementations of these algorithms rather than carrying decompression libraries as dependencies.

Summary

• Use -z or --search-zip to enable transparent decompression when searching compressed files with ripgrep.
• Supported formats include gzip, bzip2, xz, lz4, lzma, and zstd, detected via extensions defined in crates/ignore/src/default_types.rs.
• Architecture spawns external decompression commands (e.g., gzip -d -c) and pipes stdout to the search engine without temporary files, as implemented in crates/cli/src/decompress.rs.
• Security restricts automatic decompression to known formats; use --pre for custom decompression commands.
• Performance is maintained through streaming decompression directly into the core searcher.

Frequently Asked Questions

What compression formats does ripgrep support natively?

ripgrep natively supports gzip (.gz), bzip2 (.bz2), xz (.xz), lz4 (.lz4), lzma (.lzma), and zstd (.zst). The tool detects these via file extensions listed in crates/ignore/src/default_types.rs and invokes the corresponding system decompression utilities.

How does ripgrep decompress files without creating temporary files?

The decompression driver in crates/cli/src/decompress.rs spawns external commands like gzip -d -c and pipes their stdout directly into ripgrep's search engine. The core searcher accepts any Read implementation, allowing it to process the decompressed byte stream in memory without writing to disk.

Can I use a custom decompression command?

Yes, but you must use the --pre flag to specify a custom pre-processor rather than the automatic -z handling. For example, rg --pre 'custom_decompressor' pattern files/ tells ripgrep to pipe file contents through your specified command before searching, bypassing the built-in extension checking.

Why does ripgrep skip my compressed files without the -z flag?

Without the -z flag, ripgrep treats files with compressed extensions as binary data and skips their contents to avoid matching against compressed byte sequences. According to the flag definitions in crates/core/flags/defs.rs, you must explicitly enable search-zip mode to treat these files as searchable text streams.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}