How Maru Handles Closure Compilation: From Lambda to Runtime Object

Maru compiles closures by emitting a <target-function> object that stores a machine code label at compile time, then creates a runtime closure value that captures the current lexical <context> for indirect invocation.

Maru is a self-hosting Lisp system implemented by Attila Lendvai that compiles to native machine code. Understanding Maru closure compilation reveals how the system bridges high-level lambda expressions with low-level object emission and runtime context capture, ensuring lexical scoping persists across function calls.

Compile-Time Phase: Building the Closure Object

Maru’s compiler transforms a lambda expression into a full object containing both the compiled machine code and metadata for runtime instantiation. This process relies on forward declarations, label generation, and structured object emission.

Forward Declarations in emit-x86-common.l

The compiler first declares the symbols needed for closure compilation in source/compiler/emit-x86-common.l. These forward declarations ensure the assembler recognizes closure-related constructs before their full definitions:

(define *special-forms*) ; forward
(define compile-call-to-<target-function>) ; forward
(define closure-code-label) ; forward

Source: source/compiler/emit-x86-common.l#L3-L6

Emitting the Object

The core routine emit-object/<target-function> in source/compiler/emit-x86-objects.l constructs the actual closure object. It generates a unique label for the object, emits the object header, and stores the entry-point label pointing to the compiled code:

(define-function emit-object/<target-function> (name entry-label -c-)
  (let ((object-label (if name
                          (LABEL (if (feature redefinable-functions-in-target)
                                     (concat-string (symbol->string name) "__obj")
                                     name))
               (UNIQUE-LABEL "oclosure")))
    (emit-object/header type type-id -c-)
    (emit DEFLABEL object-label)
    (emit CELL entry-label)        ; the code pointer
    (when name (emit CELL name))   ; optional name for debugging
    object-label))

Source: source/compiler/emit-x86-objects.l#L57-L76

This function creates a unique label for the object (object-label) and stores entry-label—a label pointing to the compiled code—inside the object’s first cell.

Generating Code Labels with closure-code-label

When the feature emit-with-object-headers is enabled, the compiler uses closure-code-label to generate named labels that tie the code to its object:

(define-function closure-code-label (name)
  ;; KLUDGE object "names" (i.e. addresses) should be tied together
  ;; explicitly, not through giving them the "right" name
  (LABEL (concat-string (symbol->string name) "__code")))

Source: source/compiler/emit-x86-objects.l#L140-L144

If the feature is disabled, the label is simply the supplied name wrapped in GLOBAL.

The compile-expr Entry Point

The compile-expr function serves as the main entry point for lambda compilation. It assembles the code label, emits the function body, and creates the closure object:

(define-function compile-expr (self name -c-)
  (emit-comment 1 "compiling fn "name", "self)
  (let ((entry-point (if name
                        (closure-code-label name)
                        (UNIQUE-LABEL "code"))))
    (emit-expr-code self entry-point -c-)
    ;; Emit a full <target-function> object so callers can treat it as a closure
    (emit-object/<target-function> name entry-point -c-)))

Source: source/compiler/emit-x86-objects.l#L145-L156

When emit-with-object-headers is off, the same function only returns the entry point label without emitting the full object.

Calling Compiled Closures

When a closure is invoked, the compiler expands to a CALL sequence that loads the code pointer from the object header and performs an indirect call:

(gen LOAD-OOP (LITERAL program-code-offset))   ; fetch the entry label from the object
(gen CALLPTR arg-locs)                         ; indirect call through the loaded label

Source: source/compiler/emit-x86-objects.l#L124-L134

Runtime Phase: Creating Closure Values

After compilation, the Maru VM instantiates closures by capturing the current lexical environment and binding it to the compiled code object.

Closure Data Structures and Context Capture

The VM stores a context (an array of bound values) and a pointer to the compiled code. The <env> records lexical scope, with its level field incremented each time a lambda boundary is crossed:

- `level`: incremented when crossing lambda boundaries. It is needed for
  the implementation of closures capturing variables from their lexical environment.

Source: doc/vm.md#L47-L51

A closure captures the <context> at the moment the lambda is instantiated:

Closures capture the `<context>` where they are instantiated and this
is how they share variables with other closures.

Source: doc/vm.md#L60-L62

Lambda Creation in the Evaluator

During evaluation, a lambda form creates a closure object that bundles the current <context> with the compiled function object (<target-function>). The evaluator constructs this by capturing the active environment and associating it with the code pointer:

(lambda (params body)
  (let ((captured-context current-context)
        (code-pointer (lookup-compiled-function name))) ; <target-function> object
    (make-closure captured-context code-pointer)))

The relevant environment handling code lives in source/evaluator/eval.l around line 220, where a TODO notes that capturing strings into closures remains unsupported:

(lambda (str candidate) ; TODO there's no support yet for capturing str into the closure

Source: source/evaluator/eval.l#L220

When a closure is applied, the VM sets up a new <env> whose parent points to the captured context, increments level, and evaluates the function body with the supplied arguments.

Complete Example: From Source to Execution

Consider a classic closure pattern in Maru:

;; Source code written in Maru
(define (make-adder n)
  (lambda (x) (+ x n)))      ; creates a closure that captures `n`

The compilation process follows these steps:

1. compile-expr for the lambda body creates a code label make-adder__code.
2. emit-object/<target-function> builds an object:
   • First cell: pointer to make-adder__code
   • Second cell (optional): function name for debugging
3. The outer function returns a runtime closure:
   • Captured <context> = value of n
   • Pointer = object created in step 2

Execution flow:

> (define add5 (make-adder 5))
> (add5 10)
15

The VM looks up the closure’s captured context (n = 5), loads the code pointer from the <target-function> object, and invokes the compiled routine with arguments 10 and the captured 5.

Key Source Files for Maru Closure Compilation

File	Role	Link
source/compiler/emit-x86-objects.l	Emits <target-function> objects, defines closure-code-label and compile-expr (core of closure compilation)	view
source/compiler/emit-x86-common.l	Forward declarations used by the object emitter	view
source/compiler/emit-llvm-objects.l	Same logic for the LLVM backend (label generation, object emission)	view
source/evaluator/eval.l	Runtime handling of lambda creation, closure capture, and application	view
doc/vm.md	High-level description of the VM’s closure data structures	view

Summary

• Maru closure compilation generates a <target-function> object at compile time that stores a code label and optional debugging name.
• The compile-expr function in source/compiler/emit-x86-objects.l orchestrates label generation and object emission.
• At runtime, the evaluator captures the current <context> (lexical environment) and bundles it with a pointer to the compiled object.
• Closure invocation uses LOAD-OOP to fetch the code pointer from the object header followed by CALLPTR for indirect execution.
• The level field in <env> tracks lexical nesting depth to support variable capture across nested lambda boundaries.

Frequently Asked Questions

How does Maru represent a compiled closure in memory?

Maru represents a compiled closure as a <target-function> object that contains a header, a code pointer (stored as a cell referencing the compiled machine code label), and optionally a name cell for debugging. At runtime, this object is paired with a captured <context> array that holds the lexical environment values, forming a complete closure value that the VM can invoke.

What is the difference between compile-time and runtime closure handling in Maru?

At compile-time, Maru uses compile-expr and emit-object/<target-function> to generate machine code for the lambda body and emit a static object containing the code label. At runtime, the evaluator in source/evaluator/eval.l creates a dynamic closure value by capturing the current <context> and associating it with the pre-compiled <target-function> object, enabling lexical scoping across function calls.

How does Maru invoke a closure at the machine code level?

When invoking a closure, Maru generates assembly that first executes LOAD-OOP with a literal offset to fetch the code pointer from the <target-function> object header, then performs CALLPTR to make an indirect call through that loaded address. This mechanism, found in source/compiler/emit-x86-objects.l, ensures the closure executes with its captured lexical environment intact.

What limitations exist for closure capture in the current Maru implementation?

According to the source code in source/evaluator/eval.l at line 220, Maru currently has a TODO indicating that capturing string literals directly into closures is not yet supported. While the system fully supports capturing variables and contexts through the <env> level mechanism, certain literal types may require additional implementation work to be properly enclosed in the runtime closure object.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}