How Magika's Deep Learning Model Identifies File Types: A Technical Deep Dive

Magika identifies file types using a three-stage pipeline that extracts integer tokens from file headers and footers, runs them through an ONNX deep learning model, and applies confidence thresholds and label overrides to produce the final result.

Magika is Google's open-source file type identification library that uses deep learning to detect content types with high accuracy. Unlike traditional tools that rely solely on magic numbers, Magika's deep learning model analyzes file content patterns to identify file types. This article breaks down exactly how the model processes files from raw bytes to final classification, based on the source code in the google/magika repository.

Stage 1: Feature Extraction from File Headers and Footers

Magika begins by reading only the first and last block of a file, keeping the process lightweight and fast. The feature extraction logic converts raw bytes into integer tokens by stripping leading and trailing whitespace, then padding the sequence to a fixed size.

In the Python implementation, this logic resides in Magika._extract_features_from_seekable (approximately lines 210-260 of python/src/magika/magika.py). For high-performance scenarios, the identical algorithm is implemented in Rust within extract_features_async in rust/lib/src/input.rs.

The extraction process produces a fixed-size vector of integers representing the file's header and footer patterns, which serves as the input tensor for the neural network.

Stage 2: ONNX Model Inference

Once features are extracted, Magika loads the serialized deep learning model from python/src/magika/models/standard_v3_3/model.onnx. The model session is initialized in Magika._init_onnx_session, which configures the ONNX runtime with the appropriate execution providers.

The inference execution occurs in Magika._get_raw_predictions (lines 94-147 of python/src/magika/magika.py). This method feeds the integer token matrix into the ONNX graph via onnxruntime, yielding raw logits for each supported content type. The model analyzes patterns in the byte sequences to predict probabilities across hundreds of file types, from common formats like PDF and JPEG to specialized formats.

Stage 3: Post-Processing with Thresholds and Overwrite Maps

The final stage converts raw model outputs into actionable file type classifications. This happens in _get_output_label_from_dl_label_and_score, which implements a sophisticated decision logic:

First, the system selects the label with the highest logit as the model label. Then it consults the overwrite map defined in python/src/magika/models/standard_v3_3/config.min.json (loaded by _load_model_config, lines 301-329). This map can substitute specific labels with more appropriate ones—for example, treating certain HTML variants as plain TXT when the confidence is borderline.

Finally, the system applies confidence thresholds. The configuration specifies per-type thresholds and a global medium_confidence_threshold. If the prediction score falls below the threshold for the current PredictionMode (HIGH_CONFIDENCE, MEDIUM_CONFIDENCE, or BEST_GUESS), Magika falls back to a generic TXT or UNKNOWN label rather than risking a false positive.

The assembled result—including the raw label, final output label, confidence score, and overwrite rationale—is packaged into a MagikaResult object by _get_result_from_labels_and_score and returned to the caller.

Practical Code Examples

The Magika Python API provides multiple entry points for file type identification depending on your use case.

Identify a File by Path

from pathlib import Path
from magika import Magika

magika = Magika()                     # loads the default model (standard_v3_3)

result = magika.identify_path(Path("example.pdf"))

print(result.prediction.output.label) # → ContentTypeLabel.PDF

print(result.prediction.score)       # confidence score (0.99 …)

print(result.prediction.overwrite_reason)  # e.g., OverwriteReason.NONE

Identify Raw Bytes

from magika import Magika

data = b"%PDF-1.7\n%..."   # first few bytes of a PDF file

result = Magika().identify_bytes(data)

print(result.prediction.output.label)  # PDF

Stream-Based Identification

import io
from magika import Magika

with open("large_video.mkv", "rb") as f:
    stream = io.BufferedReader(f)
    result = Magika().identify_stream(stream)

print(result.prediction.output.label)  # MKV

Adjust Confidence Mode

from magika import Magika, PredictionMode

magika = Magika(prediction_mode=PredictionMode.MEDIUM_CONFIDENCE)
result = magika.identify_path("ambiguous_file.bin")
print(result.prediction.output.label)  # MAY fall back to UNKNOWN if score < medium threshold

Summary

• Feature extraction reads only the first and last blocks of a file, converting bytes to integer tokens via Magika._extract_features_from_seekable (Python) or extract_features_async (Rust).
• ONNX inference feeds tokens to the deep learning model in model.onnx, executing in Magika._get_raw_predictions to generate raw logits for each content type.
• Post-processing applies confidence thresholds and the overwrite map from config.min.json via _get_output_label_from_dl_label_and_score, falling back to generic labels when confidence is insufficient.
• The final MagikaResult object contains the output label, confidence score, and overwrite rationale, providing a complete classification result.

Frequently Asked Questions

What file formats does Magika's deep learning model support?

Magika's standard_v3_3 model supports hundreds of content types ranging from common document formats like PDF, DOCX, and HTML to multimedia formats like JPEG, MP4, and MKV, as well as programming language source files and generic text. The complete list of supported labels is defined in the model's config.min.json configuration file.

How does Magika handle files that are too small for deep learning analysis?

If a file is smaller than the required block size for feature extraction, Magika still processes it by reading the available bytes and padding the sequence with a special padding_token value. The model is trained to handle variable-length inputs, and the post-processing thresholds ensure that low-confidence predictions on minimal data fall back to appropriate generic labels like TXT or UNKNOWN.

Can I use Magika's model without the Python runtime?

Yes, Magika provides a high-performance Rust implementation that uses the same ONNX model and feature extraction logic. The Rust library in rust/lib/src/input.rs implements extract_features_async for feature extraction, and the model constants are defined in rust/lib/src/model.rs. This allows integration into systems where Python is not available or where maximum performance is required.

What is the difference between PredictionMode.HIGH_CONFIDENCE and MEDIUM_CONFIDENCE?

PredictionMode.HIGH_CONFIDENCE requires the model's prediction score to exceed a high threshold (typically 0.95) before accepting the label, otherwise falling back to UNKNOWN or TXT. PredictionMode.MEDIUM_CONFIDENCE uses a lower threshold (typically 0.50), allowing more predictions to pass through but with increased risk of misclassification. BEST_GUESS mode applies no threshold and always returns the model's top prediction regardless of confidence.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}