# How Monty Implements Its Bytecode VM Architecture: Compiler, Operations, and Execution

> Explore Monty's bytecode VM architecture implementation. Learn how its compiler transforms AST nodes into bytecode, how operations are defined, and how the stack-based VM executes code efficiently.

- Repository: [Pydantic/monty](https://github.com/pydantic/monty)
- Tags: architecture
- Published: 2026-02-16

---

**Monty uses a CPython-inspired pipeline where a compiler transforms prepared AST nodes into compact bytecode, which a stack-based virtual machine executes with support for external calls and full state snapshotting.**

The `pydantic/monty` repository implements a Python-compatible bytecode virtual machine in Rust. Understanding Monty's bytecode VM architecture reveals how it bridges high-level Python semantics with low-level execution efficiency through a classic compiler-VM pipeline designed for pausable, resumable execution.

## Monty Bytecode VM Architecture Overview

Monty's interpreter follows the classic CPython model: **source → AST → Bytecode → Stack-based VM**. The entire pipeline lives in `crates/monty/src/bytecode/`.

| Component | What it does | Key source |
|-----------|--------------|------------|
| **Compiler** | Walks the prepared AST (`PreparedNode`, `Expr`, `PreparedFunctionDef`) and emits a linear stream of opcodes plus constant tables. It also builds a list of nested `Function` objects that are later looked up by the VM. | [`crates/monty/src/bytecode/compiler.rs`](https://github.com/pydantic/monty/blob/main/crates/monty/src/bytecode/compiler.rs) |
| **Opcode enumeration** | Defines every instruction the VM can execute (e.g., `LoadConst`, `BinaryAdd`, `CallFunction`, `Await`). The enum is converted from a raw byte during dispatch. | [`crates/monty/src/bytecode/op.rs`](https://github.com/pydantic/monty/blob/main/crates/monty/src/bytecode/op.rs) |
| **Code object** | Holds the bytecode vector, constant pool, exception tables, and line-number information used for tracebacks. | [`crates/monty/src/bytecode/code.rs`](https://github.com/pydantic/monty/blob/main/crates/monty/src/bytecode/code.rs) |
| **VM core** | A stack-based interpreter that fetches opcodes, manipulates the operand stack, manages a call-frame stack, and drives the scheduler for async tasks. It can pause on external calls and be snapshotted. | [`crates/monty/src/bytecode/vm/mod.rs`](https://github.com/pydantic/monty/blob/main/crates/monty/src/bytecode/vm/mod.rs) |
| **Snapshot & Resume** | When the VM hits `ExternalCall`, `OsCall`, or `ResolveFutures` it serialises its entire state (`VMSnapshot`) together with the heap and namespaces. `Snapshot::restore` rebuilds a fresh `VM` that can continue execution. | Same file – the `VMSnapshot` struct and `VM::snapshot` / `VM::restore` methods |
| **Run-time entry point** | `MontyRun::new` parses and prepares the source, `run` (or `run_no_limits`) creates a heap, namespaces, and a `VM`, then calls `VM::run_module`. The public API also exposes `start` for iterative execution. | [`crates/monty/src/run.rs`](https://github.com/pydantic/monty/blob/main/crates/monty/src/run.rs) |

## The Compiler: From AST to Bytecode

Located in [`crates/monty/src/bytecode/compiler.rs`](https://github.com/pydantic/monty/blob/main/crates/monty/src/bytecode/compiler.rs), the **Compiler** walks prepared AST nodes and emits a linear opcode stream.

### Compilation Pipeline

The entry point `Compiler::compile_module` receives a slice of `PreparedNode` objects. It creates a `CodeBuilder`, then calls `compile_block` to walk each node. For expressions, it emits stack-manipulating opcodes like `LoadConst` and `BinaryAdd`. Control-flow nodes generate jump labels via `emit_jump` and `patch_jump`.

### Function Compilation

Function definitions trigger `compile_function_body` recursively. The body compiles first, then a `Function` record containing the compiled `Code` and metadata appends to the compiler's `functions` vector. Finally, `MakeFunction` or `MakeClosure` opcodes emit, referencing the function by its index in that vector.

At the end of the module, the compiler injects a `LoadNone`/`ReturnValue` pair so the top-level always returns a value. All emitted opcodes are stored in a `Code` object together with a constant pool (`Value`s) and exception tables.

## Bytecode Representation and Opcodes

### Code Objects

The `Code` struct in [`crates/monty/src/bytecode/code.rs`](https://github.com/pydantic/monty/blob/main/crates/monty/src/bytecode/code.rs) holds the bytecode vector, constant pool (`Value`s), exception tables, and line-number information for tracebacks.

### Opcode Definitions

[`crates/monty/src/bytecode/op.rs`](https://github.com/pydantic/monty/blob/main/crates/monty/src/bytecode/op.rs) defines the `Opcode` enumeration. Instructions include `LoadConst`, `BinaryAdd`, `CallFunction`, `Await`, `ReturnValue`, and `MakeFunction`. The VM converts raw bytes to these variants during dispatch.

## The Execution Engine: Stack-Based VM

The VM core resides in [`crates/monty/src/bytecode/vm/mod.rs`](https://github.com/pydantic/monty/blob/main/crates/monty/src/bytecode/vm/mod.rs). It implements a **stack-based interpreter** with cached frames and reference-counted heap objects.

### Core Execution Loop

The `VM::run` method implements the fetch-decode-execute cycle. It caches the current frame in a `CachedFrame` (holding a reference to the bytecode slice and instruction pointer `ip`) to avoid repeated `frames.last_mut()` lookups.

The loop fetches the next byte, maps it to an `Opcode` via `Opcode::try_from`, and dispatches through a large `match` statement. Helper macros like `fetch_u8!`, `jump_relative!`, `handle_call_result!`, and `try_catch_sync!` keep the dispatch tight and ensure the instruction pointer saves before any operation that might raise an exception or pause execution.

### Call Frames and Stack Management

The VM maintains three primary mutable structures:

- **`stack: Vec<Value>`** – The operand stack for arithmetic, calls, and data manipulation
- **`frames: Vec<CallFrame>`** – The call-frame stack, where each frame owns its instruction pointer (`ip`) and namespace index
- **`heap: &mut Heap<T>`** – Reference-counted objects (lists, dicts, strings, user objects)

Stack manipulation opcodes like `Dup` copy the top value without immediately incrementing the reference count, then increment only if the value is a `Ref`. This pattern prevents borrow conflicts with the mutable `Heap`.

### Handling External and Async Calls

When `CallFunction` encounters an external function, it synchronizes the cached `ip` back to the current frame, then invokes `exec_call_function`. The `handle_call_result!` macro processes four outcomes:

1. **`Push`** – Push result onto stack
2. **`FramePushed`** – New frame for Python function calls
3. **`External`** – Return `FrameExit::ExternalCall` with a fresh `CallId`
4. **`OsCall`** – Similar to external but for OS-level operations

For async execution, the `Await` opcode checks for `ExternalFuture`. If found, it returns `AwaitResult::Yield` containing the pending `CallId`s, causing the VM to yield `FrameExit::ResolveFutures`. The `scheduler::Scheduler` tracks tasks, their frames, and pending calls, enabling true coroutine switching.

## Snapshotting and Resumption

Monty's VM supports **full state serialization** through the `VMSnapshot` struct in [`crates/monty/src/bytecode/vm/mod.rs`](https://github.com/pydantic/monty/blob/main/crates/monty/src/bytecode/vm/mod.rs).

When the VM yields for `ExternalCall`, `OsCall`, or `ResolveFutures`, it calls `self.snapshot()`. This method moves (not clones) the entire execution state—including the operand stack (`Vec<Value>`), serialized call frames (`SerializedFrame`), exception stack, instruction pointer, and the async `Scheduler`—into a `VMSnapshot`. This transfer of ownership preserves reference counts.

To resume, `Snapshot::restore` rebuilds a fresh `VM`:

```rust
let mut vm = VM::restore(
    snapshot,               // VMSnapshot
    module_code,            // &Code of the module (needed for frames with function_id = None)
    &mut heap,
    &mut namespaces,
    &interns,
    &mut print_writer,
);
let result = vm.run()?; // continues where it left off

```

The public API (`Snapshot::run`, `Snapshot::run_pending`) abstracts this plumbing, allowing users to supply concrete return values or `ExternalFuture` objects for async resolution.

## Summary

- Monty's **bytecode VM architecture** follows the CPython model: AST → Bytecode → Stack-based execution.
- The **compiler** in [`crates/monty/src/bytecode/compiler.rs`](https://github.com/pydantic/monty/blob/main/crates/monty/src/bytecode/compiler.rs) transforms `PreparedNode` ASTs into opcode streams with constant pools and function tables.
- **Opcodes** defined in [`crates/monty/src/bytecode/op.rs`](https://github.com/pydantic/monty/blob/main/crates/monty/src/bytecode/op.rs) include stack operations (`LoadConst`, `BinaryAdd`), control flow, and async primitives (`Await`).
- The **VM** in [`crates/monty/src/bytecode/vm/mod.rs`](https://github.com/pydantic/monty/blob/main/crates/monty/src/bytecode/vm/mod.rs) implements a cached-frame dispatch loop with explicit operand stacks and call-frame management.
- **External calls** and **async execution** pause the VM, returning control to the host with a `CallId`.
- **Snapshotting** serializes the full VM state (stacks, frames, scheduler) via `VMSnapshot`, enabling resumption through `VM::restore`.

## Frequently Asked Questions

### How does Monty's bytecode compiler differ from CPython's?

Monty's compiler operates on a **prepared AST** (`PreparedNode`, `Expr`, `PreparedFunctionDef`) rather than CPython's raw syntax tree. It emits opcodes into a `CodeBuilder` with explicit jump patching (`emit_jump`, `patch_jump`) and maintains a separate `functions` vector for function definitions, emitting `MakeFunction` or `MakeClosure` opcodes that reference these by index. While the opcode semantics mirror CPython, the implementation in [`crates/monty/src/bytecode/compiler.rs`](https://github.com/pydantic/monty/blob/main/crates/monty/src/bytecode/compiler.rs) is Rust-native and designed for snapshotting support.

### What happens when the VM encounters an external function call?

When executing a `CallFunction` opcode where the target is an external function, the VM synchronizes its cached instruction pointer back to the current `CallFrame`, then invokes `exec_call_function`. The `handle_call_result!` macro detects the external call outcome and returns `FrameExit::ExternalCall` containing a freshly allocated `CallId`. The VM then calls `self.snapshot()` to serialize its state into a `VMSnapshot`, transferring ownership of the operand stack, frames, and scheduler without cloning. Control returns to the host, which can later resume execution via `VM::restore`.

### How does Monty handle async/await in the bytecode VM?

Monty implements async/await through the `Await` opcode and a `scheduler::Scheduler`. When the VM executes `Await` on an `ExternalFuture`, it returns `AwaitResult::Yield` containing the pending `CallId`s, causing the VM to yield `FrameExit::ResolveFutures`. The scheduler tracks tasks, their frames, and pending calls, enabling true coroutine switching. The VM can be snapshotted at these yield points and resumed later when the external futures resolve, making async execution fully compatible with Monty's external call model.

### Can the VM state be serialized and resumed across different processes?

Yes, through the `VMSnapshot` mechanism. When the VM yields for external calls or async resolution, `VM::snapshot()` moves the entire execution state—including the operand stack (`Vec<Value>`), serialized call frames (`SerializedFrame`), exception stack, instruction pointer, and the async `Scheduler`—into a `VMSnapshot` struct. This transfer of ownership (rather than cloning) preserves reference counts. The `Snapshot::restore` function can reconstruct a fresh `VM` instance from this snapshot, allowing execution to continue exactly where it paused. While the snapshot itself is in-memory, the design supports serialization frameworks that could enable cross-process persistence.