What Does the `core` Folder Contain in dbt‑labs/dbt‑core?

The dbt‑labs/dbt‑core repository does not contain a single top‑level core folder; instead, core functionality is distributed across multiple Rust crates, each exposing a core.rs module that handles compilation, Jinja templating, dependency graphs, and SQL execution.

The dbt‑labs/dbt‑core repository powers the popular dbt (data build tool) analytics engineering workflow. While many developers expect a monolithic core directory containing the engine’s heart, the actual architecture spreads these critical components across specialized Rust crates under the crates/ path. Understanding this distributed layout is essential for contributors debugging compilation errors or extending the framework.

No Top-Level core Directory: The Distributed Crate Architecture

Contrary to conventional repository layouts, the root of dbt‑core contains no folder named core. Instead, dbt’s rewrite in Rust organizes core logic into separate crates. Each crate exposes its central logic through a core.rs file or equivalent module entry point, allowing for modular compilation and clear separation of concerns.

The following five crates constitute the core of dbt:

• dbt-compilation – crates/dbt-compilation/src/core.rs
• dbt-jinja-ctx – crates/dbt-jinja-ctx/src/core.rs
• dbt-adapter-sql – crates/dbt-adapter-sql/src/lib.rs
• dbt-dag – crates/dbt-dag/src/mod.rs
• dbt-schemas – crates/dbt-schemas/src/schemas/

Breaking Down the Core Implementation Files

crates/dbt-compilation/src/core.rs: The Compilation Engine

This file implements the central compilation pipeline. It transforms dbt model files into executable DAG nodes by parsing macros, resolving ref() and source() calls, and generating the SQL that runs against your warehouse.

crates/dbt-jinja-ctx/src/core.rs: Jinja Context Management

This module supplies the Jinja templating environment. It injects variables like target, var, and ref into templates, enabling dynamic SQL generation during the compilation phase.

crates/dbt-adapter-sql/src/lib.rs: SQL Adapter Foundation

While not named core.rs, this file contains the foundational query execution logic. It manages database connections, transaction lifecycles, and the translation of compiled SQL into warehouse-specific commands.

crates/dbt-dag/src/mod.rs: Dependency Graph Implementation

This module builds and schedules the directed acyclic graph (DAG) that determines execution order. It ensures models run only after their dependencies complete, handling parallelism and node selection.

crates/dbt-schemas/src/schemas/: Configuration Validation

These schema definitions enforce the structure of dbt_project.yml, profiles.yml, and model contracts. They validate JSON schemas to ensure configuration files conform to expected formats before compilation begins.

How the Core Components Work Together

The distributed core modules collaborate through a specific pipeline:

1. Parsing & Rendering – dbt-jinja-ctx creates the Jinja environment and renders model files.
2. Compilation – dbt-compilation processes rendered models, resolves references, and produces execution nodes.
3. Dependency Resolution – dbt-dag constructs the graph and determines topological order.
4. Execution – dbt-adapter-sql executes the compiled SQL against the target warehouse.
5. Validation – dbt-schemas validates all configuration throughout the lifecycle.

Practical Example: Navigating the Core Flow

The following Python snippet illustrates how these Rust core modules interface through dbt’s Python API:

from dbt_core import DbtProject

# Load project configuration

project = DbtProject.from_directory("/path/to/project")

# Build Jinja context (calls dbt_jinja_ctx::core internally)

jinja_ctx = project.compile_context()

# Compile into DAG nodes (uses dbt_compilation::core)

compiled = project.compile(jinja_ctx)

# Determine execution order (dbt_dag)

execution_order = compiled.dag.topological_sort()

# Execute against warehouse (dbt_adapter_sql)

for node in execution_order:
    adapter.execute(node.rendered_sql)

Summary

• The dbt‑core repository has no single core folder at the root level.
• Core functionality resides in five specialized Rust crates under crates/.
• Key files include crates/dbt-compilation/src/core.rs and crates/dbt-jinja-ctx/src/core.rs.
• The architecture separates compilation, templating, DAG management, execution, and validation concerns.
• Contributors should target specific crate core.rs files when modifying engine behavior.

Frequently Asked Questions

Is there a core folder in the dbt‑core GitHub repository?

No, the repository uses a distributed crate architecture. Core logic is split across multiple Rust packages like dbt-compilation and dbt-jinja-ctx, each containing a core.rs module rather than a centralized directory.

What language is the dbt core written in?

The dbt engine is being rewritten in Rust (as evidenced by the crates/ directory structure), though Python bindings and APIs remain available for user interaction. The core compilation and execution logic resides in Rust source files.

Which file handles the compilation of dbt models?

The crates/dbt-compilation/src/core.rs file contains the central compilation pipeline. It parses model files, resolves references, and transforms them into executable nodes within the DAG.

How does dbt determine the order to run models?

The crates/dbt-dag/src/mod.rs module implements the dependency graph. It performs topological sorting on the compiled nodes to ensure models execute only after their referenced dependencies complete.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}