KCL Compiler Architecture: Responsibilities of the Core Components in the /crates Directory

The /crates directory houses the modular Rust implementation of the KCL compiler, where discrete crates manage distinct compilation phases—from tokenizing source text and building the AST to semantic validation, runtime evaluation, and CLI integration.

The KCL configuration language compiler follows a strictly modular architecture organized within the /crates folder. Each crate encapsulates a specific responsibility in the compilation pipeline, enabling independent testing, reuse across the Language Server Protocol (LSP) implementation, and straightforward binding generation for multi-language SDKs. Understanding these components reveals how KCL transforms declarative configuration into executable output.

Frontend: Lexing, Parsing, and AST Management

The frontend crates handle the transformation of raw KCL source code into a structured representation.

Lexical Analysis

The lexer crate (crates/lexer/src/lib.rs) performs the initial tokenization, converting raw source text into a stream of lexical tokens. This stream serves as the input for the parser, establishing the fundamental vocabulary of the language.

Syntax Parsing

The parser crate (crates/parser/src/lib.rs) consumes the token stream and constructs the Abstract Syntax Tree (AST). It implements the grammar rules that define valid KCL syntax, producing a structured tree representation of schemas, rules, and expressions.

AST Definitions and Utilities

The ast crate (crates/ast/src/lib.rs) defines the node types, token structures, and tree walker utilities used throughout the compiler. Complementing this, the ast_pretty crate (crates/ast_pretty/src/lib.rs) provides formatting and pretty-printing capabilities utilized by the kcl fmt command and debugging tools.

Semantic Analysis and Type Checking

Once the AST is constructed, the compiler validates the program's meaning and consistency.

The sema crate (crates/sema/src/ty/mod.rs) performs comprehensive semantic analysis including type checking, constraint enforcement, and symbol resolution. It ensures that schema attributes match their declared types and that validation rules are logically sound.

The query crate (crates/query/src/lib.rs) builds upon semantic analysis to provide query-style access to the AST and symbol tables. This functionality powers IDE features like "go to definition" and workspace symbol search in the LSP implementation.

Compilation Pipeline Orchestration

The compiler crate (crates/compiler/src/lib.rs) serves as the central orchestrator, chaining the discrete compilation phases into a cohesive pipeline: lexical analysis → parsing → semantic analysis → intermediate representation (IR) generation. It coordinates the frontend and analysis crates to produce executable artifacts from source code.

Supporting this, the loader crate (crates/loader/src/mod.rs) manages module resolution and the virtual file system. It resolves import paths, loads external files, and assembles compilation units before passing them to the compiler pipeline.

Runtime Execution Environment

The execution phase involves three closely related crates with distinct responsibilities.

The runtime crate (crates/runtime/src/lib.rs) provides core runtime support including value representations, memory management, and built-in functions for JSON, YAML, crypto, and other operations. It defines the primitive operations available to executing KCL programs.

The evaluator crate (crates/evaluator/src/value.rs) executes the compiled representation, evaluating expressions, functions, and schema instances against the runtime primitives. It handles the actual computation of configuration values and validation logic.

The runner crate (crates/runner/src/lib.rs) functions as the high-level entry point for program execution. It initializes the runtime context, sets up the evaluation environment, and invokes the evaluator to produce final results.

API and CLI Interfaces

Integration points for external tools and end-users reside in dedicated interface crates.

The api crate (crates/api/src/lib.rs) exposes the public API surface used by Go, Python, Java, and other language SDKs. It provides stable functions for parsing, compiling, and running KCL programs from external runtimes.

The cmd crate (crates/cmd/src/run.rs) implements the command-line interface (kcl binary). It wires user-provided arguments through the compilation and execution flow, handling subcommands like run, fmt, and vet.

Configuration management falls to the config crate (crates/config/src/vfs.rs), which processes command-line flags and .kcl.yaml configuration files to establish compiler settings.

Development Tooling

The tools crate (crates/tools/src/vet/validator.rs) aggregates ancillary development utilities including the formatter, linter, and Language Server Protocol (LSP) implementation. These tools leverage the query and semantic analysis crates to provide real-time feedback during development.

Supporting Infrastructure

Several utility crates provide cross-cutting concerns essential to the compiler's operation.

The error crate (crates/error/src/lib.rs) centralizes diagnostic definitions, error formatting, and warning management used consistently across all phases. The utils crate (crates/utils/src/lib.rs) contains general-purpose helper functions for file I/O and path handling shared throughout the workspace.

The version crate (crates/version/src/lib.rs) manages compiler version metadata. The macros crate (crates/macros/src/lib.rs) supplies procedural macros for compiler-internal boilerplate generation. Finally, the spec crate (crates/spec/src/lib.rs) maintains language specification data including versioning and feature flags.

Practical Usage Examples

Evaluating KCL Programmatically

To execute KCL code embedded in another Rust application, combine the loader, runtime, and evaluator:

use kcl_evaluator::Evaluator;
use kcl_loader::Loader;
use kcl_runtime::Runtime;

fn main() -> anyhow::Result<()> {
    let source = r#"
    schema Person {
        name: str
        age: int
    }
    p = Person {
        name = "Alice"
        age = 30
    }
    p
    "#;

    let loader = Loader::default();
    let prog = loader.load_program_from_str("example.k", source)?;
    
    let mut runtime = Runtime::default();
    let result = Evaluator::new(&mut runtime).eval_program(&prog)?;
    
    println!("Result: {}", result);
    Ok(())
}

Key files: crates/evaluator/src/value.rs, crates/loader/src/mod.rs, crates/runtime/src/lib.rs.

Parsing a KCL File

Direct AST parsing without full execution uses the parser and AST crates:

use kcl_parser::parse_file;
use kcl_ast::ast::Program;

fn parse_kcl_file(path: &str) -> anyhow::Result<Program> {
    let program = parse_file(path)?;
    Ok(program)
}

Key file: crates/parser/src/lib.rs.

Running the CLI from Code

For applications embedding KCL CLI functionality:

use kcl_cmd::run::run_file;

fn main() -> anyhow::Result<()> {
    // Equivalent to `kcl run myfile.k --output json`
    run_file("myfile.k", &["--output", "json"])
}

Key file: crates/cmd/src/run.rs.

Summary

• The lexer and parser crates form the compiler frontend, transforming source text into an AST defined by the ast crate.
• The sema crate performs type checking and validation, while query enables IDE-style analysis of the codebase.
• The compiler crate orchestrates the full pipeline, and the loader handles module resolution through a virtual file system.
• Execution relies on three layers: the runner (entry point), evaluator (expression execution), and runtime (built-in functions and memory).
• External integration occurs through the api crate (SDKs) and cmd crate (CLI), supported by config for settings management.
• The tools crate provides LSP, formatting, and linting capabilities, while error and utils supply cross-cutting infrastructure.

Frequently Asked Questions

How does the loader differ from the parser in KCL's architecture?

The loader (crates/loader/src/mod.rs) handles module resolution and assembles compilation units from the virtual file system, while the parser (crates/parser/src/lib.rs) focuses solely on transforming token streams into AST nodes. The loader orchestrates file discovery and import resolution before passing individual files to the parser for syntactic analysis.

What is the relationship between the evaluator, runner, and runtime crates?

The runner (crates/runner/src/lib.rs) serves as the high-level entry point that initializes the runtime (crates/runtime/src/lib.rs) and invokes the evaluator (crates/evaluator/src/value.rs). The runtime provides built-in functions and memory management, while the evaluator executes expressions and schema validations against these runtime primitives.

Which crate should developers use to build custom KCL language tools?

The query crate (crates/query/src/lib.rs) provides programmatic access to AST and symbol tables for analysis, while the api crate (crates/api/src/lib.rs) exposes the stable public interface used by Go, Python, and Java SDKs. For IDE functionality specifically, the tools crate (crates/tools/src/vet/validator.rs) contains the Language Server Protocol (LSP) implementation and vetting utilities.

Where is the compilation pipeline orchestrated in the KCL source code?

The compiler crate (crates/compiler/src/lib.rs) coordinates the full compilation pipeline, chaining together lexing, parsing, semantic analysis, and intermediate representation generation. It acts as the central dispatcher that invokes the lexer, parser, and sema crates in sequence to produce executable output.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}