How to Implement Custom Dimension Types in DAT Beyond Time and Categorical

To implement custom dimension types in DAT, extend the DimensionType enum in Dimension.java, define type-specific parameters in TypeParams, add validation logic to PreBuildValidator, implement SQL generation methods in SemanticAdapter, and register the type in project_schema.json.

DAT (Data Analysis Tool) is an open-source semantic layer framework that enables organizations to define consistent metrics and dimensions. While the junjiem/dat repository ships with categorical and time dimension types out of the box, the architecture supports extending the type system to handle geospatial coordinates, currency, boolean flags, or any domain-specific classification. This guide walks through the complete implementation of custom dimension types using the actual source code structure.

Understanding DAT's Dimension Architecture

The dimension system in DAT centers on the Dimension class located at dat-core/src/main/java/ai/dat/core/semantic/data/Dimension.java. This class contains the DimensionType enum (lines 37‑63), which serves as the single source of truth for supported dimension kinds. Each enum constant maps to a string value used during JSON/YAML deserialization via the fromValue method (lines 55‑58).

The Dimension class also nests a TypeParams static class that holds type-specific metadata. For example, time dimensions use TimeGranularity to specify DAY, MONTH, or YEAR rollups. When you implement custom dimension types, you extend both the enum and this parameter structure.

Step-by-Step Guide to Implement Custom Dimension Types

Step 1: Extend the DimensionType Enum

Add your new dimension type constant to the DimensionType enum in Dimension.java. The enum uses a VALUE_MAP for reverse lookup during deserialization, so adding the constant automatically makes it available to Jackson and the configuration parser.

// dat-core/src/main/java/ai/dat/core/semantic/data/Dimension.java
public enum DimensionType {
    CATEGORICAL("categorical"),
    TIME("time"),
    GEOSPATIAL("geospatial");   // <-- new custom type

    private final String value;
    private static final Map<String, DimensionType> VALUE_MAP = new HashMap<>();

    static {
        for (DimensionType type : values()) {
            VALUE_MAP.put(type.value, type);
        }
    }

    DimensionType(String value) {
        this.value = value;
    }

    @JsonValue
    public String getValue() {
        return value;
    }

    public static DimensionType fromValue(String value) {
        return VALUE_MAP.get(value);
    }
}

Step 2: Define Type-Specific Parameters

If your custom dimension requires additional metadata (such as an SRID for geospatial data or a currency code), extend the TypeParams class inside Dimension.java. The @JsonInclude(JsonInclude.Include.NON_NULL) annotation ensures that only relevant fields serialize to JSON.

// Inside Dimension.java
@Setter
@Getter
@JsonInclude(JsonInclude.Include.NON_NULL)
public static class TypeParams {
    // Existing time granularity field
    @JsonProperty("time_granularity")
    private TimeGranularity timeGranularity;

    // New field for geospatial dimensions
    @JsonProperty("srid")
    private Integer srid;

    public void setSrid(String srid) {
        this.srid = Integer.valueOf(srid);
    }
}

Step 3: Update Semantic Validation

The PreBuildValidator class enforces semantic rules on dimensions before the model builds. Located at dat-core/src/main/java/ai/dat/boot/PreBuildValidator.java, the validateDimensionEnumValues method (around line 200) checks that enum values align with the declared dimension type. Extend this logic to handle constraints specific to your custom type.

// PreBuildValidator.java – inside validateDimensionEnumValues(...)
if (dimension.getType() == Dimension.DimensionType.GEOSPATIAL) {
    // Example rule: every enum value must be a valid WKT string
    for (Dimension.EnumValue ev : dimension.getEnumValues()) {
        if (!(ev.getValue() instanceof String) ||
            !((String) ev.getValue()).matches("^POINT\\(.*\\)$")) {
            return ValidationMessage.builder()
                .level(Level.ERROR)
                .message("Invalid geospatial enum value: " + ev.getValue())
                .build();
        }
    }
}

Step 4: Implement Adapter Logic for SQL Generation

Database adapters translate dimension operations into SQL fragments. The SemanticAdapter interface (dat-core/src/main/java/ai/dat/core/adapter/SemanticAdapter.java) defines methods like applyTimeGranularity for time-based rollups. Add a new method for your custom type and implement it in each database-specific adapter (e.g., PostgreSQL, MySQL).

// SemanticAdapter.java
public interface SemanticAdapter {
    // Existing time method
    String applyTimeGranularity(String dateExpr, Dimension.TypeParams.TimeGranularity granularity);

    // New method for geospatial handling
    default String applyGeoTransform(String geomExpr, Dimension.TypeParams params) {
        // Default no-op; dialects override as needed
        return geomExpr;
    }
}

For PostgreSQL with PostGIS support:

// PostgreSqlSemanticAdapter.java
@Override
public String applyGeoTransform(String geomExpr, Dimension.TypeParams params) {
    if (params != null && params.getSrid() != null) {
        return "ST_Transform(" + geomExpr + ", " + params.getSrid() + ")";
    }
    return geomExpr;
}

Step 5: Extend the Project Schema

DAT validates project definitions against a JSON Schema located at dat-sdk/src/main/resources/schemas/project_schema.json. Add your new dimension type to the enum list for the type property to ensure configuration validation passes.

// project_schema.json
{
  "type": "object",
  "properties": {
    "dimensions": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "type": {
            "type": "string",
            "enum": ["categorical", "time", "geospatial"]
          },
          "type_params": {
            "type": "object",
            "properties": {
              "srid": { "type": "integer" }
            }
          }
        }
      }
    }
  }
}

Step 6: Use the Custom Dimension in Your Model

With the infrastructure in place, declare dimensions using your new type in SemanticModel configurations. The setDimensions logic in SemanticModel.java (lines 50‑68) automatically deserializes the enum and any custom typeParams you defined.

# Example project configuration

dimensions:
  - name: location
    type: geospatial
    type_params:
      srid: 4326
    enum_values:
      - value: "POINT(-122.42 37.77)"
        label: "San Francisco"
      - value: "POINT(-74.00 40.71)"
        label: "New York"

Key Files for Custom Dimension Implementation

• dat-core/src/main/java/ai/dat/core/semantic/data/Dimension.java – Core Dimension class and DimensionType enum (add new types here).
• dat-core/src/main/java/ai/dat/core/semantic/data/SemanticModel.java – Holds the list of dimensions; reads the enum and typeParams (lines 50‑68).
• dat-sdk/src/main/resources/schemas/project_schema.json – JSON schema that validates user‑defined dimensions; extend the "type" enum here.
• dat-core/src/main/java/ai/dat/boot/PreBuildValidator.java – Performs semantic validation; add custom checks for new types (around line 200).
• dat-core/src/main/java/ai/dat/core/adapter/SemanticAdapter.java – Interface for DB‑specific handling; add new methods for custom dimension logic.
• dat-adapters/dat-adapter-postgresql/src/main/java/ai/dat/adapter/postgresql/PostgreSqlSemanticAdapter.java – Example concrete adapter where you implement SQL generation for a new type (e.g., PostGIS functions).
• dat-core/src/test/java/... – Location for unit tests verifying the new dimension type works end‑to‑end.

Summary

• Extend the enum – Add your custom type to Dimension.DimensionType in Dimension.java to register it with the deserialization system.
• Define parameters – Use Dimension.TypeParams to store type-specific metadata like SRID or currency codes.
• Validate semantics – Extend PreBuildValidator.validateDimensionEnumValues to enforce constraints specific to your custom type.
• Generate SQL – Add methods to SemanticAdapter and implement them in database-specific adapters (e.g., PostgreSqlSemanticAdapter) to handle type-specific SQL transformations.
• Update schemas – Register the new type in project_schema.json to pass configuration validation.
• Test thoroughly – Create unit tests that instantiate models with your custom dimension and verify validation and SQL generation.

Frequently Asked Questions

What is the DimensionType enum in DAT?

The DimensionType enum is defined inside Dimension.java at dat-core/src/main/java/ai/dat/core/semantic/data/Dimension.java. It acts as the single source of truth for supported dimension kinds, currently including CATEGORICAL and TIME. Each constant maps to a string value used during JSON deserialization via the fromValue method, and the enum automatically populates a VALUE_MAP for reverse lookups.

How do I validate custom dimension values?

Custom dimension validation occurs in PreBuildValidator.java within the validateDimensionEnumValues method (around line 200). You should extend this method to check that enum values conform to your custom type's constraints. For example, if implementing a geospatial type, verify that values match valid WKT (Well-Known Text) formats like POINT(lon lat) before the model builds.

Can I use custom dimensions with any database adapter?

Yes, but you must implement support in each specific adapter. The SemanticAdapter interface defines the contract for SQL generation, and you must add methods there (e.g., applyGeoTransform) for your custom type. Then implement these methods in concrete adapters like PostgreSqlSemanticAdapter or MySqlSemanticAdapter. If an adapter lacks implementation, the default no-op behavior in the interface prevents errors but may not generate optimal SQL for your type.

Where do I define the JSON schema for custom types?

The JSON schema that validates project configurations resides at dat-sdk/src/main/resources/schemas/project_schema.json. You must add your new dimension type to the "enum" array within the dimensions.items.properties.type definition. Additionally, extend the type_params object definition to include any custom fields your dimension requires (such as srid for geospatial data) so that the SDK validates user configurations correctly before runtime.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}