Is There a Canonical 'Core' Directory in dbt? Understanding the dbt‑core Repository Layout

No, the dbt‑core repository does not contain a single top‑level core/ directory; instead, the canonical core functionality is distributed across a Rust workspace composed of multiple specialized crates under the crates/ folder.

Developers exploring the dbt‑core repository often search for a centralized "core" directory that consolidates the engine's logic. Unlike monolithic Python projects that organize code into a single package, dbt‑core uses a Rust workspace architecture where the transformation engine is split across discrete, purpose-built crates. This modular design distributes core functionality across the crates/ directory rather than isolating it in a dedicated folder.

Where Core Logic Lives in dbt‑core

According to the dbt‑core source code, the repository follows a Rust workspace pattern defined in the top‑level Cargo.toml. Rather than housing logic in a dedicated core/ folder, the project organizes its engine into separate crates under the crates/ directory. Each crate encapsulates a specific domain of the dbt engine, from Jinja templating to DAG orchestration.

This distributed architecture means that what users might consider "core" functionality is actually spread across several specialized modules. The workspace aggregation allows these components to be developed and tested independently while maintaining strict interface contracts between them.

Key Core Modules and Their Locations

While no single directory holds everything, the following crates contain the most core‑like functionality in dbt‑core:

Jinja Context Handling (crates/dbt-jinja-ctx/src/core.rs)

The dbt-jinja-ctx crate provides the runtime context that Jinja templates use during rendering. The Context and ContextBuilder structures defined here supply variables and methods available inside model files.

Compilation Engine (crates/dbt-compilation/src/core.rs)

This crate drives the compilation pipeline that transforms parsed models into executable DAG nodes. The Compiler struct here orchestrates the core transformation logic.

Schema Store (crates/dbt-schema-store/src/lib.rs)

Acting as the central metadata repository, this crate holds the canonical representation of the project's schema objects. The SchemaStore struct manages table definitions and column metadata.

DAG Orchestration (crates/dbt-dag/src/mod.rs)

The directed acyclic graph of model execution is managed here. This crate handles the dependency graph that determines execution order across the dbt pipeline.

Telemetry and Tracing (crates/dbt-tracing/src/lib.rs)

Core instrumentation for logging and performance metrics lives in this crate, providing the observability layer for the engine.

Adapter Interface (crates/dbt-adapter/src/lib.rs)

If present, this crate defines the abstract adapter API that concrete adapters (such as Snowflake or Postgres) must implement to connect to data warehouses.

Working with Core Modules

The distributed nature of the core requires importing specific crates rather than a single core package. Below are examples of how these modules are typically used within the dbt‑core codebase.

Creating a Jinja Rendering Context

To build the context used for template rendering, the dbt-jinja-ctx crate provides a builder pattern:

use dbt_jinja_ctx::core::{Context, ContextBuilder};

let ctx = ContextBuilder::new()
    .with_variable("target", target_config)
    .build()
    .expect("Failed to build Jinja context");

// Render a model template
let rendered = ctx.render_template(model_sql)?;

Running the Compilation Pipeline

The compilation engine exposes a Compiler struct that processes project files:

use dbt_compilation::core::{Compiler, CompileOptions};

let compiler = Compiler::new(project_path)?;
let result = compiler.compile(CompileOptions::default())?;
println!("Compiled {} nodes", result.nodes.len());

Accessing the Schema Store

For metadata operations, the schema store provides a centralized interface:

use dbt_schema_store::store::SchemaStore;

let store = SchemaStore::from_project(project_path)?;
let table = store.get_table("my_schema", "my_table")?;
println!("Table columns: {:?}", table.columns);

Understanding the Workspace Structure

The top‑level Cargo.toml file defines the workspace members, aggregating all crates into a unified build. This configuration eliminates the need for a physical core/ directory while maintaining logical separation of concerns. When building dbt‑core, Cargo resolves dependencies across these crates, linking the Jinja context, compilation engine, and DAG manager into a single executable.

This architecture supports modular development, allowing contributors to test and modify specific components (like the compilation engine in crates/dbt-compilation/src/core.rs) without affecting the entire codebase. The workspace manifest serves as the authority that binds these distributed modules into the cohesive dbt engine.

Summary

• The dbt‑core repository does not contain a single canonical core/ directory.
• Core functionality is distributed across a Rust workspace under the crates/ directory.
• Key engine components reside in specific crates: dbt-jinja-ctx, dbt-compilation, dbt-schema-store, and dbt-dag.
• The workspace is configured in the top‑level Cargo.toml, which aggregates all member crates.
• Developers interact with these modules through crate‑specific imports rather than a centralized core package.

Frequently Asked Questions

Why doesn't dbt‑core use a single core directory?

The project uses a Rust workspace architecture to enforce modularity and separation of concerns. By splitting functionality into discrete crates like dbt-compilation and dbt-schema-store, the codebase allows independent testing, versioning, and development of each engine component. This pattern is idiomatic for large Rust projects and replaces the monolithic "core" folder structure found in many Python projects.

How do I find the main compilation logic in dbt‑core?

The compilation engine is located in crates/dbt-compilation/src/core.rs. This file contains the Compiler struct and the CompileOptions configuration that drive the transformation of parsed models into executable DAG nodes. This is the primary entry point for understanding how dbt compiles SQL models.

What is the difference between dbt-jinja-ctx and dbt-compilation?

The dbt-jinja-ctx crate (crates/dbt-jinja-ctx/src/core.rs) handles the runtime context for Jinja templates, providing variables and methods available during SQL rendering. The dbt-compilation crate (crates/dbt-compilation/src/core.rs) manages the compilation pipeline that orchestrates the overall build process, including parsing, dependency resolution, and node generation. The context crate feeds data into the compilation process, but they serve distinct architectural purposes.

Where is the adapter interface defined in dbt‑core?

The abstract adapter API is defined in crates/dbt-adapter/src/lib.rs. This crate establishes the interface that concrete warehouse adapters (such as Snowflake, Postgres, or BigQuery) must implement to integrate with the dbt engine. While the specific implementations live in separate repositories, the core interface contract resides here.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}