How the Shader Preprocessor Works in Godot's Rendering Server

The shader preprocessor processes raw shader source through a nine-stage pipeline—stripping comments, tokenizing input, parsing directives, expanding macros, evaluating conditionals, and resolving includes—to generate GPU-ready code while preserving line numbers for precise error reporting.

The shader preprocessor is a critical component within the godotengine/godot repository that transforms high-level shader source into driver-compatible code. Located in the rendering server under servers/rendering/, this system implements a C-like macro and conditional compilation engine specifically designed for Godot's shading language.

The Nine-Stage Preprocessing Pipeline

The ShaderPreprocessor class orchestrates a deterministic sequence of transformations. Each stage maintains precise line number tracking to ensure error messages point to the correct source location.

1. Comment Stripping

The pipeline begins with CommentRemover::strip() in servers/rendering/shader_preprocessor.cpp (line 258). This stage removes both single-line // and multi-line /* … */ comment blocks while preserving the line breaks within multi-line comments. This preservation is critical for accurate error reporting, ensuring that line numbers in the final output correspond to the original source.

2. Tokenization

The stripped source feeds into Tokenizer::advance() (line 105), which breaks the input into a stream of Token objects. Each token stores both the character value and its line number. The tokenizer specifically handles line continuations—backslash followed by newline (\ \n)—by skipping the sequence and treating the two lines as one continuous line.

3. Directive Parsing

When the tokenizer encounters a # character, process_directive() (line 377) determines which specific handler to invoke. This function acts as a router, identifying directives like #define, #if, #include, and #pragma, then delegating to the appropriate specialized processor.

4. Macro Handling

Macro processing involves two distinct operations. First, process_define() (line 411) creates Define objects that store macro names, optional argument lists, and replacement bodies. Second, expand_macros() (line 997) and expand_macros_once() repeatedly walk token lines, substituting macro bodies wherever identifiers match defined macros. The system supports function-like macros with arguments and the ## concatenation operator.

5. Conditional Compilation

Directives such as #if, #ifdef, #ifndef, #elif, #else, and #endif are handled by process_if() (line 596). The preprocessor first sanitizes condition strings via expand_condition() (line 869), performs macro expansion on the condition, then evaluates the result using Godot's Expression engine. The State object maintains a stack of Branch structures to track which code blocks are currently active.

6. Include Handling

The process_include() function (line 681) resolves #include "file.gdshaderinc" directives. It loads the referenced ShaderInclude resource, checks for cyclic dependencies using the include stack in State, and recursively preprocesses the included code with a fresh ShaderPreprocessor instance. The system enforces a recursion limit of 25 levels to prevent infinite loops.

7. Pragma Handling

Currently, process_pragma() (line 782) supports only the disable_preprocessor directive, which toggles preprocessing off for the remainder of the file. This allows writing shader code that should pass through to the GPU driver unchanged.

8. Output Generation

After processing each line, expand_output_macros() (line 856) performs final macro expansion on the output buffer. This ensures that any macros defined during preprocessing but used in output lines are fully resolved before the final string assembly.

9. Region Tracking

When configured with an r_regions parameter, the preprocessor records enabled and disabled code blocks via add_region() (line 830). These Region objects store start and end line numbers along with compilation status, enabling IDE features like greyed-out inactive code and accurate diagnostics.

Core Data Structures

The preprocessor relies on several key structures defined in servers/rendering/shader_preprocessor.h:

• State: Contains all active Define objects, the include stack, conditional Branch stack, error information, and the output buffer.
• Define: Represents a macro with its argument list and body string.
• Branch: Tracks the evaluation result of each nested conditional compilation block.
• Region: Stores metadata about code blocks for editor integration, including whether the block is compiled.

Practical Code Examples

Simple Macro and Conditional Compilation

// my_shader.gdshader
#define LIGHT_COUNT 4

#if LIGHT_COUNT > 2
    uniform vec4 light_position[LIGHT_COUNT];
#else
    uniform vec4 light_position[2];
#endif

void fragment() {
    // shader logic
}

The preprocessor evaluates the #define via process_define(), then evaluates the #if condition through process_if() after expanding LIGHT_COUNT to 4. The resulting GPU code contains only the first uniform declaration.

Using #include Directives

// common.gdshaderinc
uniform sampler2D texture_albedo;

// material.gdshader
#include "common.gdshaderinc"

void fragment() {
    ALBEDO = texture(texture_albedo, UV);
}

process_include() inlines the content of common.gdshaderinc, wrapping it with @@>res://common.gdshaderinc and @@<res://common.gdshaderinc markers to delimit the included region.

Token Concatenation

#define CAT(a, b) a ## b

int CAT(my, Var) = 5;

The ## operator triggers identifier concatenation during macro expansion, producing int myVar = 5; in the final output.

Summary

• The shader preprocessor operates through nine distinct stages from comment removal to final output generation.
• Core logic resides in servers/rendering/shader_preprocessor.cpp, with key functions like process_directive(), expand_macros(), and process_include() handling specific transformations.
• The State structure maintains macro definitions, include stacks, and conditional branch tracking throughout preprocessing.
• Recursive includes are supported up to 25 levels deep with automatic cycle detection.
• The system preserves line numbers at every stage to ensure accurate error reporting in the original source files.

Frequently Asked Questions

How does Godot's shader preprocessor differ from the C preprocessor?

While Godot's shader preprocessor implements C-like syntax for #define, #if, and #include, it uses Godot's Expression engine for conditional evaluation rather than pure integer arithmetic, and it specifically tracks line numbers for error reporting. Additionally, it supports Godot-specific resource types like ShaderInclude for modular shader development.

What is the maximum recursion depth for #include directives?

The preprocessor enforces a hard limit of 25 levels of nested includes to prevent infinite recursion. If process_include() detects a cycle in the include stack or exceeds this depth, it generates an error and halts compilation.

How does the preprocessor handle syntax errors in macros?

Errors are collected in the State object's error field, which includes a stack of FilePosition objects for precise source location tracking. When expand_macros() or process_define() encounters malformed syntax, it records the error with the exact line number from the original source file, accounting for all preprocessing transformations.

Can the shader preprocessor be disabled for specific files?

Yes. Including #pragma disable_preprocessor at the beginning of a shader file causes process_pragma() to skip all subsequent preprocessing for that file, passing the remaining source directly to the GPU driver without macro expansion or conditional compilation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}