How to Add a Custom Database Adapter in DAT Using SPI: A Complete Implementation Guide

To add a custom database adapter in DAT using SPI, implement the DatabaseAdapterFactory interface, extend GenericSqlDatabaseAdapter for JDBC-based databases, and register the factory via a META-INF/services descriptor file on the classpath.

DAT (Data‑Augmented‑Toolkit) is an open-source data integration framework that discovers database connectors at runtime through Java's Service Provider Interface (SPI) mechanism. This guide demonstrates how to add a custom database adapter in DAT using SPI based on the actual implementation patterns found in the junjiem/dat repository.

Core SPI Contracts

DAT's plug-in architecture relies on two primary interfaces located in the dat-core module.

DatabaseAdapter Interface

The DatabaseAdapter interface in src/dat-core/src/main/java/ai/dat/core/adapter/DatabaseAdapter.java defines the functional contract every adapter must fulfill. It specifies methods for semantic SQL conversion, query execution, column metadata retrieval, table seeding, and result pagination.

DatabaseAdapterFactory Interface

The DatabaseAdapterFactory interface in src/dat-core/src/main/java/ai/dat/core/factories/DatabaseAdapterFactory.java serves as the SPI entry point. Each implementation provides a unique identifier via factoryIdentifier(), declares required and optional ConfigOptions, and constructs the adapter instance in create(ReadableConfig cfg).

Discovery Mechanism

DAT uses java.util.ServiceLoader to load all DatabaseAdapterFactory implementations present on the classpath. Each adapter module must ship a service descriptor file at:

META-INF/services/ai.dat.core.factories.DatabaseAdapterFactory

This text file lists the fully-qualified factory class name. For example, the PostgreSQL adapter registers itself in src/dat-adapters/dat-adapter-postgresql/src/main/resources/META-INF/services/ai.dat.core.factories.DatabaseAdapterFactory with the content:

ai.dat.adapter.postgresql.PostgreSqlDatabaseAdapterFactory

Step-by-Step: Add a Custom Database Adapter in DAT Using SPI

To add a custom database adapter in DAT using SPI for a JDBC-compatible database, follow these four implementation steps.

Step 1: Extend GenericSqlDatabaseAdapter

Rather than implementing DatabaseAdapter directly, extend GenericSqlDatabaseAdapter from src/dat-core/src/main/java/ai/dat/core/adapter/GenericSqlDatabaseAdapter.java. This base class provides generic JDBC handling for query execution, metadata extraction, and batch inserts. Override these methods for dialect-specific behavior:

• handleSpecificTypes() – Translate JDBC-specific values to Java types.
• toAnsiSqlType() – Map database types to ANSI SQL types.
• toColumnType() – Convert ANSI types back to JDBC types.
• limitClause() – Implement pagination syntax.

// src/dat-adapters/dat-adapter-mycustomdb/src/main/java/ai/dat/adapter/mycustomdb/MyCustomDbDatabaseAdapter.java
package ai.dat.adapter.mycustomdb;

import ai.dat.core.adapter.GenericSqlDatabaseAdapter;
import ai.dat.core.adapter.data.AnsiSqlType;
import javax.sql.DataSource;
import java.sql.Types;

public class MyCustomDbDatabaseAdapter extends GenericSqlDatabaseAdapter {

    public MyCustomDbDatabaseAdapter(DataSource ds) {
        super(new MyCustomDbSemanticAdapter(), ds);
    }

    @Override
    protected Object handleSpecificTypes(Object value, int columnType) {
        return value;
    }

    @Override
    public AnsiSqlType toAnsiSqlType(int columnType, String columnTypeName,
                                      int precision, int scale) {
        return switch (columnTypeName.toUpperCase()) {
            case "MYINT" -> AnsiSqlType.INTEGER;
            case "MYTEXT" -> AnsiSqlType.TEXT;
            default -> super.toAnsiSqlType(columnType, columnTypeName, precision, scale);
        };
    }

    @Override
    protected int toColumnType(String dataType) {
        if (dataType == null) return Types.VARCHAR;
        return switch (dataType.toUpperCase()) {
            case "MYINT"   -> Types.INTEGER;
            case "MYTEXT"  -> Types.LONGVARCHAR;
            default        -> Types.VARCHAR;
        };
    }

    @Override
    public String limitClause(int limit) {
        return "LIMIT " + limit;
    }
}

Step 2: Create the Factory Class

Implement DatabaseAdapterFactory to define configuration options and instantiate your adapter.

// src/dat-adapters/dat-adapter-mycustomdb/src/main/java/ai/dat/adapter/mycustomdb/MyCustomDbDatabaseAdapterFactory.java
package ai.dat.adapter.mycustomdb;

import ai.dat.core.adapter.DatabaseAdapter;
import ai.dat.core.configuration.ConfigOption;
import ai.dat.core.configuration.ConfigOptions;
import ai.dat.core.configuration.ReadableConfig;
import ai.dat.core.factories.DatabaseAdapterFactory;

import javax.sql.DataSource;
import java.time.Duration;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Set;

public class MyCustomDbDatabaseAdapterFactory implements DatabaseAdapterFactory {

    public static final String IDENTIFIER = "mycustomdb";

    public static final ConfigOption<String> URL =
            ConfigOptions.key("url").stringType().noDefaultValue()
                    .withDescription("MyCustomDB JDBC URL");

    public static final ConfigOption<String> USERNAME =
            ConfigOptions.key("username").stringType().noDefaultValue()
                    .withDescription("MyCustomDB user name");

    public static final ConfigOption<String> PASSWORD =
            ConfigOptions.key("password").stringType().noDefaultValue()
                    .withDescription("MyCustomDB password");

    public static final ConfigOption<Duration> TIMEOUT =
            ConfigOptions.key("timeout").durationType()
                    .defaultValue(Duration.ofSeconds(30L))
                    .withDescription("MyCustomDB connection timeout");

    @Override
    public String factoryIdentifier() {
        return IDENTIFIER;
    }

    @Override
    public Set<ConfigOption<?>> requiredOptions() {
        return new LinkedHashSet<>(List.of(URL, USERNAME, PASSWORD));
    }

    @Override
    public Set<ConfigOption<?>> optionalOptions() {
        return new LinkedHashSet<>(List.of(TIMEOUT));
    }

    @Override
    public DatabaseAdapter create(ReadableConfig cfg) {
        String url = cfg.get(URL);
        Duration timeout = cfg.get(TIMEOUT);
        SimpleDataSource ds = new SimpleDataSource(url, cfg.getOptional(USERNAME), cfg.getOptional(PASSWORD), timeout);
        return new MyCustomDbDatabaseAdapter(ds);
    }
}

Step 3: Register the Service Provider

Create the service descriptor file at src/main/resources/META-INF/services/ai.dat.core.factories.DatabaseAdapterFactory:

ai.dat.adapter.mycustomdb.MyCustomDbDatabaseAdapterFactory

Step 4: Deploy to the Classpath

Package your module as a JAR and add it to DAT's runtime classpath. For Maven projects, add the dependency:

<dependency>
    <groupId>ai.dat</groupId>
    <artifactId>dat-adapter-mycustomdb</artifactId>
    <version>${project.version}</version>
</dependency>

Configuration and Usage

Reference your adapter by the identifier defined in factoryIdentifier() within your dat.yaml:

datasource:
  type: mycustomdb
  url: jdbc:mycustomdb://host/db
  username: user
  password: secret

DAT locates the matching factory, passes the configuration to create(), and returns a ready-to-use DatabaseAdapter.

Summary

• Implement DatabaseAdapterFactory to define configuration schema and instantiation logic for your custom adapter.
• Extend GenericSqlDatabaseAdapter to reuse generic JDBC implementations while overriding dialect-specific methods like toAnsiSqlType() and limitClause().
• Register via META-INF/services by creating a descriptor file with your factory's fully-qualified class name.
• Deploy the JAR to the classpath so ServiceLoader discovers your implementation at runtime.
• Configure using the factory identifier in DAT's YAML configuration files.

Frequently Asked Questions

What is the difference between implementing DatabaseAdapter directly versus extending GenericSqlDatabaseAdapter?

Implementing DatabaseAdapter directly requires writing implementations for query execution, metadata extraction, and batch processing from scratch. Extending GenericSqlDatabaseAdapter from src/dat-core/src/main/java/ai/dat/core/adapter/GenericSqlDatabaseAdapter.java provides robust default implementations for these operations, allowing you to focus solely on dialect-specific overrides such as type mapping and pagination syntax.

Where should I place the META-INF/services file in my project structure?

Place the file at src/main/resources/META-INF/services/ai.dat.core.factories.DatabaseAdapterFactory. The filename must exactly match the fully-qualified interface name, and it must contain the fully-qualified name of your factory implementation class, such as ai.dat.adapter.mycustomdb.MyCustomDbDatabaseAdapterFactory, for the ServiceLoader to locate it.

Can I use a connection pool like HikariCP instead of SimpleDataSource?

Yes. While the example uses a simple DataSource wrapper for clarity, production adapters should use connection pools. The built-in PostgreSQL and MySQL adapters in the DAT codebase use HikariCP. Simply instantiate your preferred pool in the factory's create() method and pass the DataSource to your adapter constructor.

How does DAT discover my custom adapter at runtime?

DAT uses Java's java.util.ServiceLoader mechanism. During bootstrap, DAT calls ServiceLoader.load(DatabaseAdapterFactory.class) to iterate through all registered factories. The META-INF/services descriptor file tells the ServiceLoader which classes to instantiate, automatically registering your adapter without modifying DAT core code.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}