# How the LLVM JIT Compiler Works: Embedding ORC v2 in C++ Applications

> Learn how the LLVM JIT compiler works by embedding ORC v2 in C++ applications. Compile LLVM IR to machine code at runtime and get callable function pointers.

- Repository: [LLVM/llvm-project](https://github.com/llvm/llvm-project)
- Tags: deep-dive
- Published: 2026-09-11

---

**The LLVM JIT compiler uses the ORC v2 infrastructure with `LLJIT` and `LLLazyJIT` classes to compile LLVM IR into machine code at runtime, allowing you to embed just-in-time compilation into your application by creating a JIT instance, adding modules via `ThreadSafeModule`, and looking up symbols to obtain callable function pointers.**

The llvm/llvm-project repository provides a modern Just-In-Time (JIT) compilation framework built on the On-Request Compilation (ORC) v2 architecture. This system moves beyond traditional JIT approaches by treating the JIT as a dynamic linker, enabling features like lazy compilation, concurrent compilation, and code removal at runtime.

## ORC v2 Architecture and Core Components

The LLVM JIT compiler is structured around several key abstractions defined in [`llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h`](https://github.com/llvm/llvm-project/blob/main/llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h) and related headers.

### ExecutionSession and JITDylib

At the foundation lies the **`ExecutionSession`**, which owns the entire JIT session and manages symbol strings, error reporting, and synchronization. Within this session, **`JITDylib`** instances act as dynamic libraries inside the JIT—each maintains a symbol table and can declare dependencies on other JITDylibs to control symbol resolution order.

When you embed the LLVM JIT compiler, you interact with these components through higher-level wrappers. The `ExecutionSession` locks during symbol lookup to gather required definitions, then unlocks to allow concurrent materialization.

### The Layer Stack and Materialization

ORC v2 organizes compilation through a stack of **layers** that transform and compile program representations:

- **`IRCompileLayer`** – compiles LLVM IR modules to machine code
- **`IRTransformLayer`** – applies IR transformations before compilation
- **`RTDyldObjectLinkingLayer`** – links object files using the runtime dynamic linker
- **`CompileOnDemandLayer`** – enables lazy compilation by creating stubs (used by `LLLazyJIT`)

The **`MaterializationUnit`** serves as the generic compiler interface. When `JIT->lookup("foo")` is called, the system creates a query, identifies the relevant `MaterializationUnit`, and dispatches it for compilation—either immediately (eager) or deferred (lazy).

### ThreadSafeModule and Memory Management

Before adding code to the JIT, you must wrap LLVM modules in a **`ThreadSafeModule`**, defined in [`llvm/include/llvm/ExecutionEngine/Orc/ThreadSafeModule.h`](https://github.com/llvm/llvm-project/blob/main/llvm/include/llvm/ExecutionEngine/Orc/ThreadSafeModule.h). This pairs an LLVM `Module` with a `ThreadSafeContext` to guarantee safe concurrent access. The **`jitlink::JITLinkMemoryManager`** abstracts allocation of executable memory, ensuring proper permissions and lifecycle management.

## Eager vs. Lazy Compilation Strategies

The LLVM JIT compiler provides two primary embedding strategies depending on your performance requirements.

### LLJIT for Eager Compilation

**`LLJIT`** compiles symbols immediately upon lookup. When you call `lookup()` on a symbol in the main JITDylib, the system materializes the function right away, returning a ready-to-call address. This approach suits applications where startup latency matters less than consistent execution speed.

### LLLazyJIT for On-Demand Compilation

**`LLLazyJIT`** defers compilation until the first function call. When you add a module via `addLazyIRModule()`, the system creates trampolines that trigger compilation via the `CompileOnDemandLayer` on first invocation. This reduces initial load times but introduces a pause on first execution. Configure concurrent compilation threads using `setNumCompileThreads()` to mitigate this latency.

## How to Embed the LLVM JIT Compiler

Embedding requires linking against `LLVMExecutionEngine` or `LLVMOrcJIT` and following a specific initialization pattern demonstrated in [`llvm/examples/HowToUseLLJIT/HowToUseLLJIT.cpp`](https://github.com/llvm/llvm-project/blob/main/llvm/examples/HowToUseLLJIT/HowToUseLLJIT.cpp).

### Creating the JIT Instance

Use the builder pattern to construct your JIT stack. The `LLJITBuilder` automatically detects the host target and configures the layer stack:

```cpp
#include "llvm/ExecutionEngine/Orc/LLJIT.h"

auto JITOrErr = LLJITBuilder().create();
if (!JITOrErr) {
    return JITOrErr.takeError();
}
auto &JIT = *JITOrErr;

```

For lazy compilation, substitute `LLLazyJITBuilder` and optionally configure compilation threads:

```cpp
auto LazyJITOrErr = LLLazyJITBuilder()
                       .setNumCompileThreads(4)
                       .create();

```

### Loading LLVM IR Modules

Convert your LLVM IR into a `ThreadSafeModule` before adding it to the JITDylib:

```cpp
#include "llvm/ExecutionEngine/Orc/ThreadSafeModule.h"
#include "llvm/IRReader/IRReader.h"

ThreadSafeContext TSCtx(std::make_unique<LLVMContext>());
auto Mod = parseIRFile("module.ll", Err, *TSCtx.getContext());
if (!Mod) {
    return Err;
}
ThreadSafeModule TSM(std::move(Mod), std::move(TSCtx));

if (auto Err = JIT->addIRModule(std::move(TSM))) {
    return Err;
}

```

For lazy loading, call `addLazyIRModule()` instead, which creates the deferred compilation stubs.

### Looking Up and Calling JIT-Compiled Functions

Symbol resolution follows the same linking rules as static linkers, respecting visibility and weak definitions. Retrieve function pointers by name and cast them to callable types:

```cpp
auto SymOrErr = JIT->lookup("my_function");
if (!SymOrErr) {
    return SymOrErr.takeError();
}

using FuncTy = int(*)(int, int);
auto *FuncPtr = SymOrErr->getAddress().toPtr<FuncTy>();
int Result = FuncPtr(42, 100);

```

The address returned by `getAddress()` remains valid for the lifetime of the containing JITDylib or until explicitly removed via a `ResourceTracker`.

## Thread Safety and Concurrency

The ORC v2 stack supports concurrent compilation through the `ConcurrentIRCompiler` class. When you configure multiple compilation threads via `setNumCompileThreads()`, materialization units dispatch to a thread pool while the calling thread waits for the specific symbol it requested. All JIT operations are thread-safe, though you must ensure your LLVMContext is wrapped in `ThreadSafeContext` as shown above.

The JIT-as-linker model enables reliable removal of code: `ResourceTracker`s track dependencies between symbols and materialization units, allowing you to unload specific modules without destroying the entire JIT session.

## Summary

- The LLVM JIT compiler operates on the ORC v2 architecture, treating JIT compilation as dynamic linking.
- **`LLJIT`** provides eager compilation at lookup time, while **`LLLazyJIT`** defers compilation until first call via `CompileOnDemandLayer`.
- **`ExecutionSession`** manages the JIT lifecycle while **`JITDylib`** instances organize code into searchable symbol tables.
- Embed the JIT using `LLJITBuilder`, wrap modules in `ThreadSafeModule`, and retrieve functions via `lookup()` followed by `getAddress().toPtr<>()`.
- Source files [`llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h`](https://github.com/llvm/llvm-project/blob/main/llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h) and [`llvm/examples/HowToUseLLJIT/HowToUseLLJIT.cpp`](https://github.com/llvm/llvm-project/blob/main/llvm/examples/HowToUseLLJIT/HowToUseLLJIT.cpp) provide the definitive reference implementation.

## Frequently Asked Questions

### What is the difference between LLJIT and LLLazyJIT?

`LLJIT` compiles code immediately when you call `lookup()`, ensuring the function is ready before execution begins. `LLLazyJIT`, as shown in [`llvm/examples/OrcV2Examples/LLJITWithLazyReexports.cpp`](https://github.com/llvm/llvm-project/blob/main/llvm/examples/OrcV2Examples/LLJITWithLazyReexports.cpp), uses the `CompileOnDemandLayer` to create stubs that trigger compilation only when the function is first called, trading initial call latency for faster startup times.

### How does symbol lookup work in the LLVM JIT?

According to the implementation in [`llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h`](https://github.com/llvm/llvm-project/blob/main/llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h), calling `JIT->lookup("symbol")` creates a lookup query that locks the `ExecutionSession`, gathers `MaterializationUnit`s containing the requested symbol, unlocks the session, and dispatches those units for compilation. Once materialized, the address is cached and returned to the caller.

### Is the LLVM JIT compiler thread-safe?

Yes. The ORC v2 infrastructure uses `ThreadSafeContext` and `ThreadSafeModule` to protect LLVM IR, while the `ExecutionSession` manages synchronization internally. You can safely call `lookup()` from multiple threads, and when using `setNumCompileThreads()`, compilation occurs in parallel on a thread pool without manual locking.

### What headers and libraries are required to embed the LLVM JIT?

You need [`llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h`](https://github.com/llvm/llvm-project/blob/main/llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h) for the JIT classes, [`llvm/include/llvm/ExecutionEngine/Orc/ThreadSafeModule.h`](https://github.com/llvm/llvm-project/blob/main/llvm/include/llvm/ExecutionEngine/Orc/ThreadSafeModule.h) for concurrent module handling, and [`llvm/IRReader/IRReader.h`](https://github.com/llvm/llvm-project/blob/main/llvm/IRReader/IRReader.h) if parsing IR from files. Link against `LLVMOrcJIT` and `LLVMExecutionEngine` libraries, and consult [`llvm/examples/HowToUseLLJIT/HowToUseLLJIT.cpp`](https://github.com/llvm/llvm-project/blob/main/llvm/examples/HowToUseLLJIT/HowToUseLLJIT.cpp) for a complete build example.