# Graph Optimizations in ONNX Runtime: The Complete Transformer Pipeline Explained

> Discover ONNX Runtime graph optimizations. Learn how Level 1 and Level 2 rewrite rules boost transformer pipeline performance during model loading via SessionOptions.

- Repository: [Microsoft/onnxruntime](https://github.com/microsoft/onnxruntime)
- Tags: deep-dive
- Published: 2026-04-24

---

**ONNX Runtime applies a hierarchical pipeline of rewrite rules—organized into Level 1 (basic) and Level 2 (extended) optimizations—during `InferenceSession::Initialize`, which can be controlled via `SessionOptions::graph_optimization_level` and fine-tuned by disabling specific transformers.**

When you load an ONNX model into the **microsoft/onnxruntime** inference engine, the framework automatically rewrites the computational graph for maximum performance before execution begins. This transformation process, driven by the **TransformerLevel** setting in `SessionOptions`, executes a curated list of rewrite rules that eliminate redundant operations and fuse computation-heavy patterns. Understanding these graph optimizations is essential for debugging model behavior and minimizing inference latency.

## How the Graph Transformer Pipeline Works

During `InferenceSession::Initialize`, ONNX Runtime constructs a **graph transformer pipeline** that converts the raw ONNX graph into an optimized executable form. The core logic resides in `onnxruntime/core/optimizer/graph_transformer_utils.cc`, specifically within the `GenerateRewriteRules` function.

The system organizes optimizations into **four transformer levels** defined in [`include/onnxruntime/core/optimizer/graph_transformer_level.h`](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/optimizer/graph_transformer_level.h):

- **Level 1**: Basic optimizations safe for universal application
- **Level 2**: Extended optimizations that may slightly alter numerical precision
- **Level 3-4**: Reserved for future extensions (currently empty)

Each level instantiates a **RuleBasedGraphTransformer** (named "LevelX_RuleBasedTransformer") that registers specific rewrite rules with the **GraphTransformerManager** ([`onnxruntime/core/optimizer/graph_transformer_mgr.h`](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/optimizer/graph_transformer_mgr.h)). The manager executes these rules in ascending order via `ApplyTransformers` during session initialization.

## Level 1 Optimizations (Basic)

Level 1 optimizations focus on eliminating no-op operations and performing straightforward algebraic fusions. The `GenerateRewriteRules` function registers these in the "Level1_RuleBasedTransformer":

- **EliminateIdentity**: Removes Identity nodes that pass tensors through unchanged
- **EliminateSlice**: Removes unnecessary Slice operations
- **UnsqueezeElimination**: Removes redundant Unsqueeze operations
- **EliminateDropout**: Strips Dropout nodes during inference
- **ExpandElimination**: Folds constant Expand operations into shapes
- **CastElimination** and optional **CastChainElimination**: Removes unnecessary type casting (enabled via `SessionOptions::enable_cast_chain_elimination`)
- **PreShapeNodeElimination**: Removes shape computation nodes when tensor shapes are known at graph construction time
- **NoopElimination**: Removes operations with no computational effect
- **DivMulFusion**: Combines consecutive Division and Multiplication operations
- **FuseReluClip**: Merges ReLU and Clip operations when bounds align
- **GemmSumFusion**: Fuses General Matrix Multiply (GEMM) with Add operations
- **GemmTransposeFusion**: Absorbs transpose operations into GEMM kernels
- **NotWhereFusion**: Optimizes boolean Not followed by Where operations
- **ConvAddFusion**, **ConvMulFusion**, **ConvBNFusion**: Fuses Convolution with element-wise operations and BatchNorm
- **PadFusion**: Merges padding operations into preceding convolution or pooling layers
- **MatmulBNFusion**: Fuses MatMul operations with BatchNorm
- **LabelEncoderFusion**: Optimizes categorical encoding patterns

## Level 2 Optimizations (Extended)

Level 2 transformations apply more aggressive optimizations that may affect numerical precision. These rules are registered in the "Level2_RuleBasedTransformer":

- **ClipQuantFusion**: Integrates Clip operations into quantized nodes
- **ReluQuantFusion**: Merges ReLU activations into quantization operations
- **GemmTransposeFusion**: Additional GEMM transpose fusion patterns for specific hardware backends

## Configuring Optimizations via SessionOptions

You can control the optimization pipeline through the Python API by adjusting `graph_optimization_level` and disabling specific rules:

```python
import onnxruntime as ort

# Default: Maximum optimizations (Level 1 + Level 2)

sess = ort.InferenceSession("model.onnx")

# Basic optimizations only (Level 1)

opts = ort.SessionOptions()
opts.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_BASIC
sess_basic = ort.InferenceSession("model.onnx", sess_options=opts)

# Disable specific optimizer (e.g., ConvBNFusion)

opts = ort.SessionOptions()
opts.optimizers_to_disable = ["ConvBNFusion"]
sess_no_fusion = ort.InferenceSession("model.onnx", sess_options=opts)

# Disable entire transformer level (e.g., Level 2)

opts = ort.SessionOptions()
opts.transformers_to_disable = ["Level2_RuleBasedTransformer"]
sess_no_level2 = ort.InferenceSession("model.onnx", sess_options=opts)

```

The `SessionOptions` settings are parsed in `graph_transformer_utils.cc` when `GenerateRewriteRules` constructs the transformation list for the `GraphTransformerManager`.

## Execution Provider Extensions

Beyond the core pipeline, individual **Execution Providers** (EPs) register provider-specific transformers through helper functions like `RegisterGraphTransformers`. For example, CUDA-specific optimizations are implemented in `onnxruntime/core/providers/cuda/GraphTransformerHelpers.cc`. These EP-specific rules execute alongside the standard Level 1 and Level 2 optimizations, targeting hardware-specific fusion opportunities.

## Summary

- ONNX Runtime applies **graph optimizations** through a transformer pipeline during `InferenceSession::Initialize` in `onnxruntime/core/session/inference_session.cc`.
- The pipeline uses **four transformer levels** (currently utilizing Level 1 and Level 2), controlled by `SessionOptions::graph_optimization_level`.
- **Level 1** includes 18+ rewrite rules for eliminating no-ops and basic fusions like **ConvBNFusion** and **GemmSumFusion**.
- **Level 2** adds aggressive optimizations like **ClipQuantFusion** and **ReluQuantFusion**.
- Configuration happens via `SessionOptions` fields: `optimizers_to_disable` for specific rules and `transformers_to_disable` for entire levels.
- The architecture centers on `GraphTransformerManager` in [`onnxruntime/core/optimizer/graph_transformer_mgr.h`](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/optimizer/graph_transformer_mgr.h) and `GenerateRewriteRules` in `onnxruntime/core/optimizer/graph_transformer_utils.cc`.

## Frequently Asked Questions

### What is the default graph optimization level in ONNX Runtime?

By default, ONNX Runtime uses `TransformerLevel::MaxLevel`, which enables all available optimizations (currently Level 1 and Level 2). This corresponds to `GraphOptimizationLevel.ORT_ENABLE_ALL` in the Python API and is applied automatically when you create an `InferenceSession` without modifying `SessionOptions`.

### How can I disable a specific fusion like ConvBNFusion?

You can disable individual rewrite rules by setting the `optimizers_to_disable` field in `SessionOptions` to a list containing the exact optimizer name. For example: `opts.optimizers_to_disable = ["ConvBNFusion"]`. This list is parsed in `graph_transformer_utils.cc` and excludes those rules from the `RuleBasedGraphTransformer` instances created for the session.

### Where are the graph transformer levels defined in the source code?

The `TransformerLevel` enum is defined in [`include/onnxruntime/core/optimizer/graph_transformer_level.h`](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/optimizer/graph_transformer_level.h), specifying `Level1`, `Level2`, `Level3`, `Level4`, and `MaxLevel`. The actual rules for each level are instantiated in `onnxruntime/core/optimizer/graph_transformer_utils.cc` within the `GenerateRewriteRules` function, which creates the `RuleBasedGraphTransformer` objects for each active level.

### Do execution providers add their own graph optimizations?

Yes. Execution Providers register additional provider-specific transformers through functions like `RegisterGraphTransformers` in their respective directories (e.g., `onnxruntime/core/providers/cuda/GraphTransformerHelpers.cc` for CUDA). These EP-specific optimizations complement the core Level 1 and Level 2 rules managed by the `GraphTransformerManager`, targeting hardware-specific patterns that the generic optimizer cannot detect.