Where Are Core Features Located in the dbt-labs/dbt-core Repository?

The core features of dbt-core reside in the top-level crates/ directory, distributed across three primary Rust crates—dbt-compilation, dbt-jinja-ctx, and dbt-docs-core—rather than a single monolithic folder.

The dbt-labs/dbt-core repository powers the popular data transformation tool used by analytics engineers worldwide. Unlike earlier Python-based architectures, the modern dbt-core codebase is organized as a Rust workspace composed of discrete crates. Understanding where core features live requires navigating the crates/ directory structure where the compilation engine, templating context, and documentation generator are implemented.

Core Features Folder Structure in dbt-core

The repository organizes functionality using Rust's crate-based architecture. The crates/ folder at the repository root contains all modular components, with core logic concentrated in specific compilation and context crates that handle the complete dbt workflow.

dbt-compilation Crate

The dbt-compilation crate serves as the central orchestration engine. Located at crates/dbt-compilation/, it handles project loading, incremental caching, and resolution of models and macros. The primary entry point is crates/dbt-compilation/src/core.rs, which exposes the DbtLoadedProject struct that drives the entire compilation pipeline.

dbt-jinja-ctx Crate

The dbt-jinja-ctx crate manages the Jinja templating environment essential for SQL rendering. Found at crates/dbt-jinja-ctx/, this crate constructs the execution context containing macros, variables, and adapter-specific functions. Its core implementation resides in crates/dbt-jinja-ctx/src/core.rs.

dbt-docs-core Crate

The dbt-docs-core crate powers documentation generation. Located at crates/dbt-docs-core/, it processes Parquet artifacts to produce the HTML documentation accessed via dbt docs generate. The main library entry point is crates/dbt-docs-core/src/lib.rs.

How Core Components Interact

The core crates work sequentially to transform project files into executable SQL. The workflow follows a specific pipeline from loading through adapter initialization, with each stage defined in the compilation crate's core module.

Project Loading and Hydration

The process begins with DbtLoadedProject::load in crates/dbt-compilation/src/core.rs. This method reads project configurations, resolves dependencies, and optionally reuses cached state for incremental compilation. It accepts parameters including CompilationConfig, LoadArgs, and cancellation tokens to support async operations.

Jinja Environment Creation

After loading, create_jinja_env constructs the templating environment. This method builds a JinjaEnv instance containing all project macros and variables required for SQL compilation. It operates on the loaded project state and resolver configuration.

Resolution and DAG Building

The resolve method executes the parser and resolver pipeline. This stage produces a ResolverState containing the fully-resolved Directed Acyclic Graph (DAG) of models, macros, and sources. The method handles dependency resolution and reference detection across the entire project.

Adapter Initialization

Finally, init_adapter instantiates the concrete database adapter (such as DuckDB, Snowflake, or Postgres). This method prepares the connection layer that will execute the compiled SQL against your data warehouse, accepting the resolved state and Jinja environment as inputs.

Working with Core APIs

Developers interacting with dbt-core programmatically use these specific APIs from the compilation crate. The following examples demonstrate the typical initialization flow used by the CLI and test harnesses.

Loading a project with optional incremental cache:

let loaded = DbtLoadedProject::load(
    config,                     // CompilationConfig
    load_args,                  // LoadArgs (paths, deps, etc.)
    invocation_args,            // Jinja invocation args
    type_ops_factory,           // Arc<dyn TypeOpsFactory>
    adapter_factory,            // Arc<dyn AdapterFactory>
    None,                       // No previous project → full load
    None,                       // No extra tracing config
    &cancellation_token,       // CancellationToken
    loader_hooks,               // LoaderHooks (hooks for plugins)
    jinja_factory,              // JinjaFactory
).await?;

Creating the Jinja environment:

let jinja_env = loaded.create_jinja_env(
    &resolver_state,
    &io_args,
    &invocation_args,
    &cancellation_token,
)?;

Resolving the project to build the DAG:

let (resolved_state, jinja_env) = loaded.resolve(
    resolve_args,
    &invocation_args,
    cache_state.as_ref(),
    &cancellation_token,
    jinja_listener_factory,
    resolver_hooks,
).await?;

Initializing the database adapter:

let adapter = loaded.init_adapter(
    &resolved_state,
    None,                     // No replay mode
    &jinja_env,
    Some(schema_store),       // Optional schema store for introspection
    &cancellation_token,
    None,                     // No Sidecar client
    Execute::Local,           // Run locally
)?;

Summary

• Core features reside in the crates/ directory as discrete Rust crates rather than a single "core" folder
• dbt-compilation handles project loading, resolution, and adapter initialization via crates/dbt-compilation/src/core.rs
• dbt-jinja-ctx provides the templating environment through crates/dbt-jinja-ctx/src/core.rs
• dbt-docs-core manages documentation generation from Parquet artifacts at crates/dbt-docs-core/src/lib.rs
• The CLI binary at crates/dbt-cli/src/main.rs wires these components together to implement the dbt command interface

Frequently Asked Questions

Is there a single "core" folder in dbt-labs/dbt-core?

No. The repository uses a Rust workspace structure where core functionality is distributed across multiple crates within the crates/ directory. The main compilation logic lives in dbt-compilation, while supporting core features exist in dbt-jinja-ctx and dbt-docs-core.

What programming language is dbt-core written in?

The modern dbt-core is primarily written in Rust, organized as a Cargo workspace with multiple crates. This architecture replaces the earlier Python implementation, offering improved performance and memory safety for compilation and execution tasks.

How does the CLI interact with core features?

The CLI binary located at crates/dbt-cli/src/main.rs imports and orchestrates the core crates. It calls DbtLoadedProject methods from dbt-compilation to load projects, resolve dependencies, and execute SQL against adapters, providing the command-line interface for dbt commands.

Where is the entry point for compilation logic?

The primary entry point is crates/dbt-compilation/src/core.rs, which defines the DbtLoadedProject struct. This file contains the load(), create_jinja_env(), resolve(), and init_adapter() methods that constitute the core compilation pipeline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}