# What Heuristic Magika Uses for Very Small Files: Inside the 8-Byte Threshold

> Discover Magika's 8-byte heuristic for small files. Learn how it efficiently classifies files bypassing neural networks for faster detection.

- Repository: [Google/magika](https://github.com/google/magika)
- Tags: internals
- Published: 2026-04-16

---

**When a file is smaller than 8 bytes, Magika bypasses its neural network and applies a simple heuristic that classifies valid UTF-8 bytes as `txt` and everything else as `unknown`.**

Google's Magika content-type detection system uses a lightweight fallback mechanism for files too small to feed into its deep-learning model. When a file contains fewer than 8 bytes—the default `min_file_size_for_dl` threshold defined in the Rust core—Magika triggers a heuristic that inspects the raw bytes for UTF-8 validity rather than attempting model inference.

## The 8-Byte Threshold Trigger

The decision to use the heuristic starts with a size check in [`rust/lib/src/input.rs`](https://github.com/google/magika/blob/main/rust/lib/src/input.rs). Magika reads the file into a feature vector where positions correspond to byte indices. If the file length is less than `min_file_size_for_dl` (8 bytes by default), the value at index 7 remains the special **padding token**, signaling the model cannot be used.

The code checks this condition before feature extraction:

```rust
if features[config.min_file_size_for_dl - 1] != config.padding_token {
    // File is large enough → use the ML model
    return Ok(FeaturesOrRuled::Features(Features(features)));
}

```

When this check fails, execution falls through to the heuristic path.

## The UTF-8 Validation Heuristic

For files 8 bytes or smaller, Magika applies a brutally simple content-type rule in the `extract()` function within [`rust/lib/src/input.rs`](https://github.com/google/magika/blob/main/rust/lib/src/input.rs). It attempts to decode the first block of bytes as UTF-8 text.

**If the bytes decode successfully as UTF-8**, Magika returns the content type **`txt`**, assuming the file is plain text. **If decoding fails**, it returns **`unknown`**, indicating binary or unrecognized data.

The implementation uses Rust's `std::str::from_utf8`:

```rust
let content_type = match std::str::from_utf8(&first_block) {
    Ok(_) => ContentType::Txt,      // Valid UTF-8 → plain text
    Err(_) => ContentType::Unknown, // Not UTF-8 → unknown type
};

```

This binary classification avoids the computational overhead of the neural model while providing a reasonable guess for extremely small files.

## Configuration and Defaults

The threshold constant is defined in [`rust/lib/src/model.rs`](https://github.com/google/magika/blob/main/rust/lib/src/model.rs) as part of the model configuration structure. The default value of **8 bytes** (`min_file_size_for_dl: 8`) represents the minimum input size the deep-learning model can effectively process.

Documentation in [`website-ng/src/content/docs/core-concepts/how-magika-works.md`](https://github.com/google/magika/blob/main/website-ng/src/content/docs/core-concepts/how-magika-works.md) confirms this behavior, stating that "if the file is too small for the model (e.g., under ~8 bytes), Magika uses simple heuristics to return a generic answer like `txt` or `unknown`."

## Practical Examples

### Python Client

When using the official Python client, the heuristic triggers automatically for files under 8 bytes:

```python
from magika import Magika

magika = Magika()

# 5-byte plain-text file

with open("tiny.txt", "wb") as f:
    f.write(b"Hello")

print(magika.run("tiny.txt").output)   # → txt

# 5-byte binary file

with open("tiny.bin", "wb") as f:
    f.write(b"\x00\xff\x01\x02\x03")

print(magika.run("tiny.bin").output)   # → unknown

```

### Rust Implementation

Directly invoking the feature extraction in Rust demonstrates the heuristic path:

```rust
use magika::input::extract_async;
use magika::input::AsyncFile;

#[tokio::main]
async fn main() {
    // File smaller than 8 bytes
    let result = extract_async(AsyncFile::open("tiny.bin").await.unwrap())
        .await
        .unwrap();

    match result {
        FeaturesOrRuled::Ruled(content_type) => {
            println!("Heuristic result: {}", content_type); 
            // Prints "txt" for UTF-8, "unknown" otherwise
        }
        FeaturesOrRuled::Features(_) => {
            println!("Model inference path (should not happen for tiny files)");
        }
    }
}

```

## Summary

- **Magika uses an 8-byte threshold** (`min_file_size_for_dl`) to determine when a file is too small for neural-network inference.
- Files ≤8 bytes trigger a **UTF-8 validation heuristic** in [`rust/lib/src/input.rs`](https://github.com/google/magika/blob/main/rust/lib/src/input.rs) instead of the ML model.
- **Valid UTF-8** bytes are classified as **`txt`**; **invalid UTF-8** bytes are classified as **`unknown`**.
- This fallback ensures Magika returns fast, deterministic results for edge-case files without loading the deep-learning model.

## Frequently Asked Questions

### What is the minimum file size for Magika's neural network?

The minimum file size is **8 bytes**, defined by the `min_file_size_for_dl` constant in [`rust/lib/src/model.rs`](https://github.com/google/magika/blob/main/rust/lib/src/model.rs). Files smaller than this threshold bypass the neural network entirely and use the UTF-8 heuristic instead.

### How does Magika classify a 5-byte text file?

Magika reads the 5 bytes and attempts to decode them as UTF-8. If the decoding succeeds—as it would for ASCII characters like "Hello"—the file is classified as **`txt`**. This determination happens in the `extract()` function within [`rust/lib/src/input.rs`](https://github.com/google/magika/blob/main/rust/lib/src/input.rs).

### Why does Magika return "unknown" for some tiny files?

The **`unknown`** classification occurs when the byte sequence cannot be interpreted as valid UTF-8. This typically indicates binary data or non-text encodings. Rather than guessing incorrectly, Magika's heuristic conservatively labels these files as unknown when the UTF-8 validation fails.

### Can I configure the threshold for small file detection?

The 8-byte threshold is hardcoded as a default in the model configuration, but the Rust implementation allows custom configurations when initializing the Magika instance. However, the heuristic logic itself—classifying valid UTF-8 as `txt` and invalid as `unknown`—remains fixed for any file falling below the minimum size threshold.