# Optimize fzf Performance with Large Input Lists: Chunking, Caching, and Parallelism Strategies

> Speed up fzf with large lists. Learn chunking, caching, and parallelism strategies. Tune --threads --no-sort --no-cache for millions of lines.

- Repository: [Junegunn Choi/fzf](https://github.com/junegunn/fzf)
- Tags: performance
- Published: 2026-03-01

---

**fzf processes massive datasets by breaking input into 1000-line chunks, caching partial results up to defined limits, and utilizing parallel worker Goroutines, which you can tune via `--threads`, `--no-sort`, and `--no-cache` to handle millions of lines efficiently.**

To optimize fzf performance with large input lists, you must understand how the junegunn/fzf repository streams data through its internal pipeline. The tool implements a chunked architecture that bounds memory usage while providing configurable knobs to trade matching accuracy for speed and disable expensive features like sorting and preview execution.

## How fzf Processes Large Inputs: The Chunking Architecture

### ChunkList and ChunkCache Implementation

In [`src/core.go`](https://github.com/junegunn/fzf/blob/main/src/core.go), fzf creates a `ChunkList` that owns a `ChunkCache` to manage memory efficiently. The input stream is divided into chunks of at most `chunkSize` items, defined in [`src/constants.go`](https://github.com/junegunn/fzf/blob/main/src/constants.go) lines 41-44 as **1000 lines per chunk**. This design prevents the matcher from loading the entire dataset into memory at once.

### Query Result Caching Limits

Each chunk's results are cached up to `queryCacheMax` entries, calculated as `chunkSize/5` (approximately 200 entries) according to [`src/constants.go`](https://github.com/junegunn/fzf/blob/main/src/constants.go) lines 48-50. However, the final merger stage deliberately avoids caching for very large lists, with `mergerCacheMax` capped at 100,000 entries ([`src/constants.go`](https://github.com/junegunn/fzf/blob/main/src/constants.go) lines 51-53), preventing memory exhaustion during the merge phase in [`src/merger.go`](https://github.com/junegunn/fzf/blob/main/src/merger.go).

## The Matching Algorithm and Parallelism Strategy

### Algorithm Selection Based on Input Size

The matching logic in [`src/algo/algo.go`](https://github.com/junegunn/fzf/blob/main/src/algo/algo.go) implements two strategies. For small inputs, it uses an **O(n·m) dynamic programming algorithm** (`FuzzyMatchV2`). When the product of input length and pattern length exceeds the pre-allocated slab capacity, or when the pattern exceeds **1000 runes**, the system falls back to a **greedy O(n) linear-time version** (lines 46-52). This fallback ensures that extremely large inputs do not trigger exponential slowdowns.

### Worker Pool and Thread Control

Parallelism is managed in [`src/matcher.go`](https://github.com/junegunn/fzf/blob/main/src/matcher.go), where the `Matcher` struct spawns worker Goroutines based on `Options.Threads` (lines 55-58). By default, fzf uses the number of CPU cores, but you can override this with the `--threads` flag. Increasing workers allows multiple chunks to be processed simultaneously, significantly reducing latency for datasets with millions of entries.

## Command-Line Flags to Optimize Performance

### Disabling Expensive Processing

Several flags reduce computational overhead when searching massive lists:

- **`--threads=N`** — Explicitly sets the number of parallel workers in [`src/matcher.go`](https://github.com/junegunn/fzf/blob/main/src/matcher.go). Use values between 4 and 16 for datasets exceeding 10⁶ lines.
- **`--no-sort`** — Skips the **O(n log n)** post-search sorting step defined in the options struct ([`src/options.go`](https://github.com/junegunn/fzf/blob/main/src/options.go) line 685). Use when you only need the best match or accept native input order.
- **`--no-cache`** — Disables the per-chunk `ChunkCache` ([`src/options.go`](https://github.com/junegunn/fzf/blob/main/src/options.go) line 689), reducing memory pressure for one-off searches of gigantic lists.
- **`--no-preview`** — Prevents the preview command from executing for every candidate ([`src/options.go`](https://github.com/junegunn/fzf/blob/main/src/options.go) line 693), avoiding costly subprocess spawns.

### UI and Display Limits

Restricting the interface reduces redraw overhead:

- **`--height=30%`** or **`--max-height=40`** — Limits the number of UI rows drawn, shrinking line-wrapping calculations in [`src/terminal.go`](https://github.com/junegunn/fzf/blob/main/src/terminal.go).
- **`--no-multi`** — Disables multi-selection mode, removing bookkeeping for selected indices when you only need a single result.

## Implementation Examples

### Optimized Bash Command

For a list of several million lines, combine parallelism with disabled overhead features:

```bash
cat huge_list.txt | fzf \
  --threads=4 \
  --no-sort \
  --no-cache \
  --no-preview \
  --height=30%

```

### Programmatic Go API Configuration

When embedding fzf programmatically, set the corresponding fields in the `Options` struct from [`src/options.go`](https://github.com/junegunn/fzf/blob/main/src/options.go):

```go
opts := &fzf.Options{
    Threads:   4,          // src/matcher.go worker Goroutines
    NoSort:    true,       // src/options.go line 685
    NoCache:   true,       // src/options.go line 689
    NoPreview: true,       // src/options.go line 693
    Height:    "30%",      // UI height limit
}
exitCode, err := fzf.Run(opts)

```

## Key Source Files for Performance Tuning

Understanding these files helps diagnose bottlenecks:

| File | Performance Relevance |
|------|---------------------|
| [`src/constants.go`](https://github.com/junegunn/fzf/blob/main/src/constants.go) | Defines `chunkSize` (1000), `queryCacheMax` (≈200), and `mergerCacheMax` (100,000) thresholds that control memory usage. |
| [`src/cache.go`](https://github.com/junegunn/fzf/blob/main/src/cache.go) | Implements `ChunkCache` – the per-chunk result store that can be disabled with `--no-cache`. |
| [`src/algo/algo.go`](https://github.com/junegunn/fzf/blob/main/src/algo/algo.go) | Contains the fallback logic from O(n·m) DP to O(n) greedy matching when inputs exceed slab capacity or 1000 runes. |
| [`src/matcher.go`](https://github.com/junegunn/fzf/blob/main/src/matcher.go) | Spawns worker Goroutines based on `Options.Threads`; controls parallel chunk processing. |
| [`src/merger.go`](https://github.com/junegunn/fzf/blob/main/src/merger.go) | Merges chunk results and deliberately excludes large mergers from caching (line 158). |
| [`src/core.go`](https://github.com/junegunn/fzf/blob/main/src/core.go) | Orchestrates the pipeline: chunk creation, matching, and UI setup. |
| [`src/options.go`](https://github.com/junegunn/fzf/blob/main/src/options.go) | Parses `--threads`, `--no-sort`, `--no-cache`, and other performance flags into the `Options` struct. |
| [`src/terminal.go`](https://github.com/junegunn/fzf/blob/main/src/terminal.go) | Handles UI redraws; affected by `--height` and `--max-height`. |

## Summary

To optimize fzf performance with large input lists:

- **Chunking**: fzf processes input in 1000-line chunks defined in [`src/constants.go`](https://github.com/junegunn/fzf/blob/main/src/constants.go), keeping memory bounded.
- **Parallelism**: Use `--threads=N` to spawn multiple workers in [`src/matcher.go`](https://github.com/junegunn/fzf/blob/main/src/matcher.go) for concurrent chunk processing.
- **Algorithm fallback**: The matcher automatically switches from O(n·m) DP to O(n) greedy in [`src/algo/algo.go`](https://github.com/junegunn/fzf/blob/main/src/algo/algo.go) when inputs exceed safe thresholds.
- **Memory control**: Disable per-chunk caching with `--no-cache` and avoid large merger caching (limited to 100,000 entries in [`src/constants.go`](https://github.com/junegunn/fzf/blob/main/src/constants.go)) to prevent memory exhaustion.
- **UI overhead**: Reduce redraw costs with `--no-sort`, `--no-preview`, and `--height` limits defined in [`src/options.go`](https://github.com/junegunn/fzf/blob/main/src/options.go) and [`src/terminal.go`](https://github.com/junegunn/fzf/blob/main/src/terminal.go).

## Frequently Asked Questions

### Why does fzf slow down with millions of lines?

fzf divides input into 1000-line chunks to limit memory usage, but the final merge stage and sorting overhead grow with input size. By default, fzf caps the merger cache at 100,000 entries ([`src/constants.go`](https://github.com/junegunn/fzf/blob/main/src/constants.go) lines 51-53) and uses an O(n log n) sort, which becomes expensive for very large datasets. Disabling sort (`--no-sort`) and reducing UI height mitigates this.

### How does the --threads flag improve performance?

The `--threads` flag controls the number of worker Goroutines spawned in [`src/matcher.go`](https://github.com/junegunn/fzf/blob/main/src/matcher.go) (lines 55-58). Each worker processes chunks independently, allowing fzf to utilize multiple CPU cores. For datasets exceeding one million lines, setting `--threads=4` or higher reduces latency by parallelizing the matching work across chunks.

### Should I disable caching for one-time searches?

Yes. The per-chunk query cache (`ChunkCache` in [`src/cache.go`](https://github.com/junegunn/fzf/blob/main/src/cache.go)) stores up to 200 results per chunk to speed up repeated queries. For one-time scans of gigantic lists, this cache consumes memory without providing benefit. Use `--no-cache` (mapped to `Options.NoCache` in [`src/options.go`](https://github.com/junegunn/fzf/blob/main/src/options.go) line 689) to disable it and reduce memory pressure.

### What is the difference between the DP and greedy matching algorithms?

fzf uses an O(n·m) dynamic programming algorithm (`FuzzyMatchV2`) for small inputs where the product of line length and pattern length fits within a pre-allocated slab. When this product exceeds the slab capacity or the pattern exceeds 1000 runes, fzf falls back to an O(n) greedy algorithm ([`src/algo/algo.go`](https://github.com/junegunn/fzf/blob/main/src/algo/algo.go) lines 46-52). This fallback ensures predictable performance for very large inputs but may sacrifice some fuzzy matching quality for speed.