How to Customize LLVM's Pass Pipeline: A Complete Guide to Optimization Passes

LLVM's pass pipeline is orchestrated by the PassBuilder class, which constructs hierarchical pass managers and executes analysis and transformation passes at module, function, and loop levels, while providing extension point callbacks and textual parsing APIs to inject custom optimization passes programmatically or via the command line.

LLVM's optimization framework centers on a modular pass pipeline architecture that allows compiler developers to fine-tune how intermediate representation (IR) is analyzed and transformed. The modern "new pass manager" system, implemented primarily in PassBuilder located at llvm/include/llvm/Passes/PassBuilder.h and llvm/lib/Passes/PassBuilder.cpp, enables precise control over optimization sequences through both C++ APIs and command-line interfaces. Understanding how to customize LLVM's pass pipeline is essential when targeting specific performance characteristics or integrating proprietary transformation passes.

Architecture of the LLVM Pass Pipeline

The PassBuilder Orchestration Layer

The PassBuilder class serves as the central factory for constructing standard optimization pipelines and parsing textual pipeline specifications. According to the LLVM source code, PassBuilder provides methods such as buildFunctionSimplificationPipeline(), buildModuleOptimizationPipeline(), and parsePassPipeline() to assemble pass sequences. The builder stores a TargetMachine* pointer (optional), a set of PipelineTuningOptions (controlling loop interleaving, vectorization, etc.), and optional Profile-Guided Optimization (PGO) settings to tailor the pipeline to specific targets.

Hierarchical Pass Managers

LLVM organizes passes into a hierarchy of managers that operate at different IR granularities. The core pass managers declared in llvm/IR/PassManager.h and related headers include:

• ModulePassManager – Processes entire modules
• CGSCCPassManager – Handles call-graph strongly connected components (declared in llvm/Analysis/CGSCCPassManager.h)
• FunctionPassManager – Operates on individual functions
• LoopPassManager – Targets loop nests specifically
• MachineFunctionPassManager – Works on machine-level IR

Each manager coordinates with Analysis Managers that store analysis results and enable on-demand recomputation via cross-registration through crossRegisterProxies(). The specific registration methods include registerModuleAnalyses(), registerFunctionAnalyses(), and registerLoopAnalyses().

Extension Points and Callbacks

The PassBuilder exposes extension points through callback vectors that allow targets, plugins, or users to inject custom passes at well-defined moments in the optimization sequence. Key registration methods defined in PassBuilder.h include:

• registerPipelineStartEPCallback() – Insert passes before the default optimization sequence
• registerPeepholeEPCallback() – Inject passes during peephole optimization
• registerVectorizerStartEPCallback() – Add transformations before vectorization
• registerScalarOptimizerLateEPCallback() – Target late-stage scalar optimizations
• registerOptimizerLastEPCallback() – Append passes at the very end of optimization

Customizing Optimization Passes in C++

To customize the pass pipeline programmatically, instantiate a PassBuilder, register the required analyses, and utilize extension point callbacks to insert custom passes. The following example demonstrates adding a custom function pass at the start of the pipeline:

#include "llvm/Passes/PassBuilder.h"
#include "llvm/IR/LLVMContext.h"
#include "llvm/IR/Module.h"

int main() {
  llvm::LLVMContext Ctx;
  std::unique_ptr<llvm::Module> M = /* load or create a module */;

  // Create a PassBuilder (no TargetMachine for a generic build)
  llvm::PassBuilder PB;

  // Register default analyses
  llvm::ModuleAnalysisManager MAM;
  PB.registerModuleAnalyses(MAM);

  // Build the default O2 optimisation pipeline
  llvm::OptimizationLevel OptLevel = llvm::OptimizationLevel::O2;
  llvm::ModulePassManager MP = PB.buildPerModuleDefaultPipeline(OptLevel);

  // ---- Insert a custom pass ----
  // Example: add a simple FunctionPass that prints function names
  struct PrintFuncNamesPass : llvm::PassInfoMixin<PrintFuncNamesPass> {
    llvm::PreservedAnalyses run(llvm::Function &F,
                                llvm::FunctionAnalysisManager &) {
      llvm::errs() << "Function: " << F.getName() << "\n";
      return llvm::PreservedAnalyses::all();
    }
  };

  // Register the custom pass at the start of the pipeline
  PB.registerPipelineStartEPCallback(
      [&](llvm::ModulePassManager &MPM, llvm::OptimizationLevel) {
        llvm::FunctionPassManager FPM;
        FPM.addPass(PrintFuncNamesPass());
        // Insert the function‑pass manager into the module pipeline
        MPM.addPass(std::move(FPM));
      });

  // Run the (now extended) pipeline
  MP.run(*M, MAM);
}

When adding a function-level pass, wrap it in a FunctionPassManager and insert it via the callback; the builder automatically handles nesting the managers correctly. The registerPipelineStartEPCallback() method (see lines 45-50 in PassBuilder.h) prepends your passes before the standard optimization sequence begins.

Customizing Passes via the Command Line

LLVM's opt utility accepts textual pipeline descriptions via the -passes flag, utilizing the same parser implemented in PassBuilder.cpp. This approach enables rapid experimentation without recompiling LLVM:

# Run the standard O2 pipeline with an extra custom pass

opt -passes="module(function(print-func-names),default<O2>)" \
    -pass-plugin=./MyPlugin.so input.bc -o output.bc

The textual syntax parsed by parsePassPipeline() supports nesting managers and passing parameters to passes. The default<O2> token expands to the built-in O2 optimization pipeline defined in buildPerModuleDefaultPipeline(), while function(print-func-names) inserts the custom pass into the function pass manager. Pass names may include parameters using angle brackets, such as loop-unroll<unroll-count=4>.

Tuning Pipeline Behavior with PipelineTuningOptions

The PassBuilder exposes a PipelineTuningOptions object that controls high-level optimization behaviors such as vectorization and unrolling. Modify these options before constructing the pipeline to disable or enable specific optimization categories:

llvm::PassBuilder PB;
llvm::PipelineTuningOptions PTO;
PTO.LoopVectorization = false;   // disable loop vectorisation
PTO.LoopUnrolling = true;        // keep loop unrolling enabled
PB = llvm::PassBuilder(nullptr, PTO);   // re‑create with custom options

These tuning options affect how buildPerModuleDefaultPipeline() constructs the pass sequence, allowing you to create custom optimization levels without modifying individual pass registrations.

Summary

• PassBuilder (defined in llvm/include/llvm/Passes/PassBuilder.h) serves as the central API for constructing and customizing LLVM's pass pipeline.
• The pipeline uses hierarchical pass managers (ModulePassManager, FunctionPassManager, etc.) that coordinate with analysis managers to cache and reuse analysis results.
• Use extension point callbacks such as registerPipelineStartEPCallback() and registerVectorizerStartEPCallback() to inject custom passes at specific optimization stages.
• The textual pipeline syntax accepted by opt -passes and parsePassPipeline() enables command-line customization using strings like default<O2> and module(function(pass-name)).
• PipelineTuningOptions provides high-level control over optimization categories including loop vectorization and unrolling without requiring source code modifications.

Frequently Asked Questions

What is the difference between the old and new pass managers in LLVM?

The legacy pass manager relied on explicit pass ordering and manual analysis invalidation, while the modern "new pass manager" (as implemented in llvm/include/llvm/IR/PassManager.h) uses a hierarchical structure of PassManager templates with automatic analysis management and on-demand invalidation. The new system provides better compile-time performance and more flexible customization through PassBuilder APIs.

How do I register a custom pass at a specific point in the pipeline?

Use the appropriate extension point callback from PassBuilder.h. For example, call registerVectorizerStartEPCallback() to insert passes immediately before vectorization runs, or registerPipelineStartEPCallback() to prepend passes to the beginning of the pipeline. Pass a lambda that receives a reference to the relevant pass manager (e.g., ModulePassManager&) and add your pass using addPass().

Can I disable specific optimization passes without modifying LLVM source code?

Yes. Use the textual pipeline syntax with opt to construct a custom pass sequence excluding specific passes, or set PipelineTuningOptions flags (such as LoopVectorization = false) when building the pipeline programmatically. For fine-grained control, parse a custom pipeline string using parsePassPipeline() instead of calling buildPerModuleDefaultPipeline().

What file contains the default optimization pipeline definitions?

The default pipeline implementations are located in llvm/lib/Passes/PassBuilder.cpp. This file contains the logic for buildPerModuleDefaultPipeline(), buildFunctionSimplificationPipeline(), and other standard sequences that construct the O1, O2, and O3 optimization pipelines based on the OptimizationLevel enum defined in llvm/include/llvm/Passes/OptimizationLevel.h.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}