How bat Loads Syntax Definitions: A Deep Dive into the Implementation

bat loads syntax definitions by lazily deserializing a pre-compiled binary SyntaxSet that is either embedded in the binary or cached on disk, using the syntect library for parsing and highlighting.

The sharkdp/bat repository implements syntax highlighting through a sophisticated lazy-loading architecture. When you invoke bat to display a file, the program does not parse raw syntax definition files at runtime. Instead, it relies on a highly optimized binary format that enables near-instantaneous syntax detection and highlighting.

Overview of the Syntax Loading Architecture

bat delegates all syntax parsing and highlighting to the syntect library. Rather than distributing raw .sublime-syntax files and parsing them on every execution, bat pre-compiles the entire collection into a compressed binary blob called syntaxes.bin. This binary is either embedded directly into the bat executable or stored in a user cache directory.

The loading process follows a four-stage pipeline:

1. Build-time compilation – All .sublime-syntax files are parsed and serialized into assets/syntaxes.bin
2. Runtime source selection – bat chooses between the embedded binary or a cached file on disk
3. Lazy deserialization – The binary is decompressed and converted into a SyntaxSet only when first needed
4. Syntax resolution – The SyntaxSet is queried by file name, extension, or content to determine the appropriate highlighter

Build-Time Compilation of Syntax Definitions

Generating the syntaxes.bin Binary

During the build process, bat executes build/syntax_mapping.rs as a Cargo build script. This script collects all bundled .sublime-syntax files, parses them using syntect, and writes the resulting SyntaxSet to assets/syntaxes.bin using bincode serialization with optional flate2 compression.

This approach eliminates runtime parsing overhead. When you install bat, the syntax definitions are already in an optimized binary format ready for immediate use.

Runtime Loading and Deserialization

The SerializedSyntaxSet Enum

At runtime, bat uses the SerializedSyntaxSet enum defined in src/assets/serialized_syntax_set.rs to abstract over two possible sources:

• FromBinary – Contains the raw bytes embedded in the executable via include_bytes!("../assets/syntaxes.bin")
• FromFile – Points to a syntaxes.bin file in the user's cache directory

The enum provides a deserialize() method that handles both sources uniformly, decompressing the data if necessary and returning a fully formed syntect::parsing::SyntaxSet.

Lazy Deserialization with OnceCell

The HighlightingAssets struct in src/assets.rs manages the actual loading. It stores the SerializedSyntaxSet and a OnceCell that caches the deserialized SyntaxSet after first use.

When you request syntax highlighting, the code calls get_syntax_set():

pub fn get_syntax_set(&self) -> Result<&SyntaxSet> {
    self.syntax_set_cell
        .get_or_try_init(|| self.serialized_syntax_set.deserialize())
}

This pattern ensures that bat pays the deserialization cost only once, and only if syntax highlighting is actually requested. The first file you view triggers the decompression and parsing of the binary, while subsequent files reuse the cached SyntaxSet.

Accessing Syntax Definitions at Runtime

Once loaded, the SyntaxSet provides methods to identify the correct syntax for any input. The HighlightingAssets struct wraps these with higher-level helpers:

• get_syntax_for_path() – Examines the file path, extension, and name to find a match
• find_syntax_by_extension() – Looks up syntax by file extension
• find_syntax_by_name() – Searches by syntax name (e.g., "Rust")

These methods consult the deserialized SyntaxSet to return a SyntaxReference that the printer components use to apply highlighting.

Practical Usage Examples

Library Usage

When using bat as a library, you can load the integrated syntax definitions and query them directly:

use bat::assets::HighlightingAssets;
use bat::syntax_mapping::SyntaxMapping;

fn main() -> anyhow::Result<()> {
    // Load the integrated syntax set (lazy-deserialized on first call)
    let assets = HighlightingAssets::from_binary();

    // Example: get the Rust syntax definition
    let syntax_ref = assets
        .get_syntax(Some("Rust"), &mut bat::input::OpenedInput::new(), &SyntaxMapping::new())?
        .syntax;

    println!("Syntax name: {}", syntax_ref.name);
    Ok(())
}

This code triggers the lazy deserialization path in src/assets.rs when get_syntax() is first called.

Command-Line Usage

When you run bat from the command line, the syntax detection happens automatically:

# bat will parse the filename, extensions and first line to find a syntax

bat src/main.rs          # uses the Rust syntax

bat Dockerfile           # matches the filename mapping

bat unknown.txt          # falls back to first-line detection or plain text

Internally, bat calls HighlightingAssets::get_syntax_for_path (see src/assets.rs lines 151-186) which uses the lazy SyntaxSet described above.

Cache Directory Configuration

You can enable a cache directory to speed up subsequent starts:

export BAT_CACHE_DIR=$HOME/.cache/bat
bat some_file.rs   # on first run the syntax set is written to $BAT_CACHE_DIR/syntaxes.bin

On subsequent executions, bat loads from this file via HighlightingAssets::from_cache (lines 73-78) rather than extracting the embedded binary, reducing memory pressure and improving startup time.

Key Source Files and Implementation Details

File	Role
src/assets/serialized_syntax_set.rs	Defines SerializedSyntaxSet and the actual binary deserialization logic using bincode and flate2.
src/assets.rs	Central struct HighlightingAssets; lazy loads the SyntaxSet via OnceCell, provides lookup helpers like get_syntax_for_path and find_syntax_by_token.
build/syntax_mapping.rs	Build-time script that compiles the full set of syntax files into assets/syntaxes.bin.
src/syntax_mapping.rs	Holds user-configurable mapping rules for file-name to syntax resolution.
src/printer.rs / src/pretty_printer.rs	Consumer components that use HighlightingAssets to apply syntax highlighting while printing.

All source links point to the master branch on GitHub at https://github.com/sharkdp/bat/blob/master/.

Summary

• bat uses the syntect library for all syntax highlighting operations.
• Syntax definitions are pre-compiled at build time into a compressed binary syntaxes.bin rather than parsed at runtime.
• The SerializedSyntaxSet enum abstracts between embedded binaries and cached files on disk.
• Deserialization is lazy: the SyntaxSet is only loaded when first requested via HighlightingAssets::get_syntax_set(), cached in a OnceCell for subsequent reuse.
• Users can optionally use BAT_CACHE_DIR to store syntax assets on disk, reducing memory usage and improving startup performance.

Frequently Asked Questions

How does bat achieve fast startup times despite having hundreds of syntax definitions?

bat avoids parsing raw syntax files at startup by shipping a pre-compiled binary blob. The syntaxes.bin file is generated at build time using build/syntax_mapping.rs, which uses syntect to parse all .sublime-syntax files and serialize them with bincode. At runtime, bat only deserializes this binary on first use, storing the result in a OnceCell to eliminate repeated overhead.

Can I use custom syntax definitions with bat?

Yes, though the process requires rebuilding the syntax cache. You can place custom .sublime-syntax files in the appropriate directory and run bat cache --build. This invokes the same build-time logic found in build/syntax_mapping.rs to regenerate the syntaxes.bin file, incorporating your custom definitions into the SerializedSyntaxSet that bat loads at runtime.

What is the difference between the embedded binary and the cache directory approach?

The embedded binary (FromBinary variant of SerializedSyntaxSet) is included directly in the bat executable via include_bytes! and is always available. The cache directory approach (FromFile) stores syntaxes.bin on disk at a location specified by BAT_CACHE_DIR. The cached approach reduces memory pressure and can improve startup time after the first run, as bat reads from the filesystem rather than extracting embedded data into memory.

How does bat determine which syntax to use for a file?

bat uses a multi-step resolution process implemented in HighlightingAssets::get_syntax_for_path (lines 151-186 in src/assets.rs). First, it checks explicit user mappings from SyntaxMapping. Then it attempts to match the file name, extension, or first line content against the definitions in the loaded SyntaxSet. If no match is found, it falls back to plain text highlighting.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}