# How ProjectBuilder Builds and Validates a DAT Project: A Deep Dive into the junjiem/dat SDK

> Discover how junjiem/dat ProjectBuilder builds and validates DAT projects. Learn about its seven-step pipeline, fingerprinting, and semantic validation for artifact persistence.

- Repository: [Junjie.M/dat](https://github.com/junjiem/dat)
- Tags: deep-dive
- Published: 2026-03-05

---

**The `ProjectBuilder` class orchestrates a seven-step incremental pipeline that loads DAT project configuration, computes content fingerprints to detect file changes, and executes comprehensive semantic validation via `PreBuildValidator` before persisting artifacts to the content store.**

The `ProjectBuilder` located at `ai.dat.boot.ProjectBuilder` serves as the primary entry point for transforming DAT project source files into executable artifacts within the [junjiem/dat](https://github.com/junjiem/dat) repository. According to the source code, this builder implements an intelligent incremental build system that only validates and rebuilds when source files change, optimizing development workflows for data analytics projects while ensuring semantic correctness through rigorous pre-build checks.

## The Seven-Step Build Pipeline in ProjectBuilder

The `ProjectBuilder.build()` method in [`dat-sdk/src/main/java/ai/dat/boot/ProjectBuilder.java`](https://github.com/junjiem/dat/blob/main/dat-sdk/src/main/java/ai/dat/boot/ProjectBuilder.java) orchestrates the entire construction process through seven distinct phases that balance performance with thoroughness:

1. **Load project configuration**: Calls `ProjectUtil.loadProject` to parse [`dat_project.yaml`](https://github.com/junjiem/dat/blob/main/dat_project.yaml) into a `DatProject` model (lines 45-50).
2. **Compute fingerprint**: Generates a content hash of all source files using `ProjectUtil.contentStoreFingerprint` to detect changes (lines 51-53).
3. **Load previous state**: Retrieves the last build's `SchemaFileState` list via `BuildStateManager.loadBuildState` (lines 53-55).
4. **Detect file changes**: Compares current hashes with stored state using `FileChangeAnalyzer.analyzeChanges` (lines 55-57).
5. **Validate semantics**: If changes exist, runs `PreBuildValidator.validate` to enforce configuration sanity and SQL correctness (lines 58-61).
6. **Update content store**: Persists new file hashes and compiled artifacts via `ContentStoreManager.updateStore` (lines 62-64).
7. **Log completion**: Emits build completion status (lines 65-66).

For forced rebuilds, `ProjectBuilder.forceRebuild()` first invokes `cleanState()` to clear stored fingerprints before executing the standard pipeline, ensuring a clean slate regardless of previous build history.

## How PreBuildValidator Validates DAT Project Semantics

The `PreBuildValidator` class in [`dat-sdk/src/main/java/ai/dat/boot/PreBuildValidator.java`](https://github.com/junjiem/dat/blob/main/dat-sdk/src/main/java/ai/dat/boot/PreBuildValidator.java) enforces semantic correctness through multiple validation layers that catch errors before they reach the execution phase:

- **Configuration validation**: `FactoryUtil.validateFactoryOptions` verifies required options exist and optional parameters have correct types (lines 54-59).
- **Template rendering**: `JinjaTemplateUtil.render` processes Jinja templating in model definitions using provided variables (lines 64-70).
- **Database adapter creation**: `ProjectUtil.createDatabaseAdapter` instantiates concrete adapters (MySQL, PostgreSQL, DuckDB) based on project configuration (lines 76-78).
- **Raw SQL validation**: `validateModelSql` executes `SELECT 1 FROM (<model>)` to verify SQL syntax parses correctly on the target database (lines 17-26).
- **Semantic model validation**: `validateSemanticModelSql` uses `SemanticModelUtil.semanticModelSql` to translate models into runnable SQL, then executes `WITH ... SELECT 1 FROM ...` guard queries (lines 54-72).
- **Dimension enum validation**: `validateDimensionEnumValues` checks that declared enum values exist in underlying data and that distinct counts remain below safe thresholds (lines 200-236).
- **Data type verification**: `validateDataTypes` retrieves column metadata and ensures manually declared types match actual database column types (lines 97-108).
- **Auto-completion**: `autoCompleteDataTypes` fetches ANSI SQL types from the database for elements lacking declared `dataType` and writes them back into the semantic model (lines 124-136).

All validation failures throw `ValidationException` with human-readable error lists, aborting the build before `ContentStoreManager` persists any artifacts.

## Code Example: Building a DAT Project with ProjectBuilder

The following Java example demonstrates the public API for building a DAT project:

```java
import ai.dat.boot.ProjectBuilder;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Map;

public class BuildDemo {
    public static void main(String[] args) throws Exception {
        // 1️⃣ Project root (the directory that contains `dat_project.yaml`)
        Path projectPath = Paths.get("./my-dat-project").toAbsolutePath();

        // 2️⃣ Create the builder
        ProjectBuilder builder = new ProjectBuilder(projectPath);

        // 3️⃣ Optional runtime variables – they are injected into Jinja templates
        Map<String, Object> variables = Map.of(
                "tenantId", 42,
                "userId",   1001
        );

        // 4️⃣ Run an incremental build (validation runs only if files changed)
        builder.build(variables);

        // 5️⃣ Force a full rebuild ignoring previous state
        // builder.forceRebuild(variables);
    }
}

```

This implementation mirrors the public API documented in `ProjectBuilder` and shows how runtime variables flow into the validation and rendering phase.

## Summary

- **ProjectBuilder** orchestrates a seven-step pipeline that loads configuration, detects file changes via content hashing, and conditionally validates before persisting artifacts.
- **PreBuildValidator** enforces semantic correctness through SQL syntax checks, database adapter validation, Jinja template rendering, and optional data type verification.
- The build process is **incremental by default**, only running expensive validation when `FileChangeAnalyzer` detects modifications, with `forceRebuild()` available for clean builds.
- All validation errors throw **ValidationException** before any artifacts are written, ensuring the content store never contains invalid state.

## Frequently Asked Questions

### What triggers a validation in ProjectBuilder?

Validation runs only when `FileChangeAnalyzer.analyzeChanges` detects added, modified, or deleted files by comparing current content fingerprints against the previous build state stored by `BuildStateManager`. If no changes exist, the build skips validation and completes immediately.

### How does ProjectBuilder detect file changes?

The builder generates a cryptographic hash of all source files using `ProjectUtil.contentStoreFingerprint` during the load phase. It then retrieves the previous build's `SchemaFileState` list via `BuildStateManager.loadBuildState` and delegates comparison logic to `FileChangeAnalyzer.analyzeChanges` to identify specific changes.

### What happens if validation fails during the build?

Any validation failure in `PreBuildValidator.validate` throws a `ValidationException` containing a human-readable list of errors. The build aborts immediately before `ContentStoreManager.updateStore` persists any artifacts, ensuring the project state remains consistent and invalid configurations never reach the content store.

### Can I force a full rebuild without incremental checks?

Yes. `ProjectBuilder.forceRebuild()` removes the stored build state by calling `cleanState()` before executing the standard build pipeline. This forces the builder to treat all files as changed, running full validation and regeneration regardless of previous fingerprints.