# How bat Handles Non-Printable Characters: A Deep Dive into the Source Code

> Discover how bat visualizes non-printable characters using a configurable replacement system. Learn to represent control codes, tabs, spaces, and newlines with symbols via the --show-all flag. Deep dive into the source code.

- Repository: [David Peter/bat](https://github.com/sharkdp/bat)
- Tags: deep-dive
- Published: 2026-03-06

---

**`bat` visualizes non-printable characters through a configurable replacement system that converts control codes, tabs, spaces, and newlines into readable symbols when the `--show-all` flag is enabled.**

The `bat` command-line tool enhances standard `cat` output with syntax highlighting and Git integration, but its handling of **non-printable characters** requires special consideration for terminal safety. When displaying files containing binary data, control sequences, or whitespace characters, `bat` employs a sophisticated preprocessing pipeline defined in its Rust source code to ensure every byte is rendered visibly without breaking the terminal.

## The Architecture Behind Non-Printable Character Handling

The replacement logic centers on three tightly-coupled components in the `bat` codebase: configuration flags, notation selection, and the preprocessor implementation.

### Configuration Flags in `Config::show_nonprintable`

The primary gatekeeper is the **`show_nonprintable`** boolean flag defined in [`src/config.rs`](https://github.com/sharkdp/bat/blob/main/src/config.rs). This flag is populated by the command-line argument `--show-all` (or `-A`) in [`src/bin/bat/app.rs`](https://github.com/sharkdp/bat/blob/main/src/bin/bat/app.rs) at line 387. When enabled, this flag triggers the replacement pipeline for every line processed by the printer.

### Notation Selection with `Config::nonprintable_notation`

Users can choose between two visualization schemes via the **`nonprintable_notation`** enum defined in [`src/nonprintable_notation.rs`](https://github.com/sharkdp/bat/blob/main/src/nonprintable_notation.rs):

- **Caret notation**: Displays control characters as `^@`, `^A` through `^Z`, `^[`, `^\`, `^]`, `^^`, `^_`, and `^?` for DEL
- **Unicode notation**: Uses Unicode Control Pictures block (U+2400–U+243F) such as `␀`, `␁`, `␊`, `␡`

### The `replace_nonprintable` Preprocessor

The core transformation logic resides in **`replace_nonprintable`** within [`src/preprocessor.rs`](https://github.com/sharkdp/bat/blob/main/src/preprocessor.rs) (lines 59–119). This function accepts a raw byte slice, tab width configuration, and notation preference, returning a `String` where every non-printable byte has been substituted with its visual representation.

## How bat Processes Non-Printable Characters Step-by-Step

When `bat` prints a file with `--show-all` enabled, the following pipeline executes for every line:

1. **Line Acquisition**: `Printer::print_line` in [`src/printer.rs`](https://github.com/sharkdp/bat/blob/main/src/printer.rs) receives a `&[u8]` buffer containing the raw file bytes.

2. **Conditional Replacement**: At lines 146–155 in [`src/printer.rs`](https://github.com/sharkdp/bat/blob/main/src/printer.rs), the code checks `self.config.show_nonprintable`. If true, it invokes `replace_nonprintable` with the line buffer, tab width, and notation settings; otherwise, it writes the buffer directly.

   ```rust
   if self.config.show_nonprintable {
       let line = replace_nonprintable(
           line_buffer,
           self.config.tab_width,
           self.config.nonprintable_notation,
       );
       write!(handle, "{line}")?;
   } else {
       // direct write
   }
   ```

3. **Byte-by-Byte Processing**: Inside `replace_nonprintable`, the function iterates through the buffer, attempting UTF-8 decoding via `try_parse_utf8_char`. Based on the resulting character or byte value, it applies specific substitutions.

4. **Character Classification and Substitution**:
   - **Space** (`' '`): Replaced with `·` (middle dot)
   - **Tab** (`'\t'`): Expanded to a visual tab stop using box-drawing characters (`├──┤`) respecting the configured `tab_width`
   - **Line feed** (`'\n'`): Replaced with caret (`^J`) or Unicode (`␊`) notation, followed by the actual newline
   - **Control codes** (`0x00`–`0x1F`): Mapped to caret notation (`^@` through `^_`) or Unicode Control Pictures (`␀`–`␟`)
   - **Delete** (`0x7F`): Rendered as `^?` or `␡`
   - **Printable ASCII**: Emitted unchanged
   - **Other bytes**: Escaped using Rust's `escape_unicode` format (e.g., `\u{00E4}`)

5. **Output Generation**: The function returns a complete `String` containing the visualized line, which `Printer::print_line` writes to the output handle.

## Character Mapping Reference

The following table details how `bat` visualizes each character class when `--show-all` is enabled:

| Character Type | Byte Value | Caret Notation | Unicode Notation |
|----------------|------------|----------------|------------------|
| Null | `0x00` | `^@` | `␀` |
| Control A–Z | `0x01`–`0x1A` | `^A`–`^Z` | `␁`–`␚` |
| Escape | `0x1B` | `^[` | `␛` |
| File Separator | `0x1C` | `^\` | `␜` |
| Group Separator | `0x1D` | `^]` | `␝` |
| Record Separator | `0x1E` | `^^` | `␞` |
| Unit Separator | `0x1F` | `^_` | `␟` |
| Space | `0x20` | `·` | `·` |
| Tab | `0x09` | Box-drawing | Box-drawing |
| Line Feed | `0x0A` | `^J` + LF | `␊` + LF |
| Delete | `0x7F` | `^?` | `␡` |
| Non-ASCII | `0x80`+ | `\u{XXXX}` | `\u{XXXX}` |

## Practical Examples

### Command-Line Usage

To visualize all non-printable characters in a file using the default Unicode notation:

```bash
bat -A example.txt

```

To use caret notation instead:

```bash
bat --show-all --nonprintable-notation=caret binary-file.dat

```

### Programmatic Usage in Rust

You can leverage `bat`'s preprocessor directly in Rust code:

```rust
use bat::preprocessor::{replace_nonprintable, NonprintableNotation};

fn main() {
    let raw_bytes = b"\tHello\x00World\n";
    
    let visualized = replace_nonprintable(
        raw_bytes,
        4,                                    // tab width
        NonprintableNotation::Caret,         // or NonprintableNotation::Unicode
    );
    
    println!("{}", visualized);
}

```

**Output (caret notation):**

```

├──┤Hello^@World^J

```

**Output (unicode notation):**

```

├──┤Hello␀World␊

```

## Summary

- **`bat` does not filter non-printable characters**; it visualizes them deterministically when `--show-all` (`-A`) is enabled.
- **Configuration is controlled** by `Config::show_nonprintable` and `Config::nonprintable_notation` in [`src/config.rs`](https://github.com/sharkdp/bat/blob/main/src/config.rs).
- **Two notation systems** are available: caret notation (`^M`) and Unicode Control Pictures (`␍`), defined in [`src/nonprintable_notation.rs`](https://github.com/sharkdp/bat/blob/main/src/nonprintable_notation.rs).
- **Core logic** resides in `replace_nonprintable` within [`src/preprocessor.rs`](https://github.com/sharkdp/bat/blob/main/src/preprocessor.rs), which handles byte-by-byte substitution, tab expansion, and UTF-8 decoding.
- **Integration** occurs in [`src/printer.rs`](https://github.com/sharkdp/bat/blob/main/src/printer.rs) where `Printer::print_line` conditionally invokes the preprocessor based on configuration.

## Frequently Asked Questions

### How do I show all non-printable characters in bat?

Use the `--show-all` flag (or `-A` for short). This enables the replacement logic that visualizes spaces as `·`, tabs as box-drawing characters, and control codes using either caret or Unicode notation depending on your configuration.

### What is the difference between caret and Unicode notation in bat?

**Caret notation** represents control characters using the `^` symbol followed by a letter or punctuation mark (e.g., `^G` for bell, `^J` for line feed). **Unicode notation** uses the Unicode Control Pictures block (e.g., `␇` for bell, `␊` for line feed). You can select between them using the `--nonprintable-notation` flag.

### Does bat modify the actual file content when showing non-printable characters?

No, `bat` only modifies the **displayed output**. The `replace_nonprintable` function in [`src/preprocessor.rs`](https://github.com/sharkdp/bat/blob/main/src/preprocessor.rs) creates a new visual representation of the data for terminal output, leaving the original file bytes untouched. This is a read-only transformation that occurs during the printing pipeline in [`src/printer.rs`](https://github.com/sharkdp/bat/blob/main/src/printer.rs).

### How does bat handle UTF-8 characters that are not ASCII?

For bytes outside the ASCII printable range (`0x20`–`0x7E`) that form valid UTF-8 sequences, `bat` attempts to decode them using `try_parse_utf8_char` within the preprocessor. If decoding succeeds, the character is emitted unchanged. If the bytes are invalid UTF-8 or represent non-printable control codes, they are escaped using Rust's `escape_unicode` format (e.g., `\u{00E4}`) or replaced with the configured notation symbols.