Architecture of DBX's Agent Service for JDBC Databases: Snowflake, Hive, and BigQuery

DBX implements a modular, JSON-RPC-based agent service that unifies JDBC database access through a common core and database-specific driver implementations.

The t8y2/dbx repository provides a plug-in-style agent architecture that enables seamless interaction with diverse JDBC-compatible databases including Snowflake, Apache Hive, and Google BigQuery. This design abstracts database-specific complexities behind a uniform JSON-RPC interface, allowing client applications to communicate with any supported data source using standardized method calls.

Core Architecture Components

The DBX agent service centers on a shared common core located in agents/common, which provides generic JDBC handling and communication infrastructure. This modular approach allows database-specific drivers to reside in isolated modules under agents/drivers/* while inheriting standardized behavior.

AbstractJdbcAgent Base Class

The AbstractJdbcAgent class in agents/common/src/main/java/com/dbx/agent/AbstractJdbcAgent.java serves as the foundation for all database agents. It manages connection lifecycles, implements generic metadata helpers, and provides default implementations for SQL execution and result conversion.

Concrete agent implementations must override three key abstract methods:

• driverClass() – Returns the fully-qualified JDBC driver class name (e.g., net.snowflake.client.jdbc.SnowflakeDriver).
• buildJdbcUrl(ConnectParams) – Constructs database-specific JDBC URLs.
• resultValue(ResultSet, int, int) – Safely converts SQL result values to Java objects.

The base class also supplies utilities like JdbcIdentifiers for proper identifier quoting and handles the heavy lifting for connection management.

JsonRpcServer and Communication Layer

Communication between the DBX client and agent processes flows through JsonRpcServer in agents/common/src/main/java/com/dbx/agent/JsonRpcServer.java. This lightweight server binds to STDIN/STDOUT, accepting JSON-RPC requests and dispatching them to the concrete agent implementation.

Each driver JAR contains an entry point that instantiates the server:

public static void main(String[] args) {
    new JsonRpcServer(new SnowflakeAgent()).run();
}

This architecture enables process isolation and language-agnostic client interaction, as the DBX client communicates via JSON rather than direct Java APIs.

Shared Data Models

The common core defines database-agnostic POJOs that standardize schema metadata across drivers. These include DatabaseInfo, TableInfo, ColumnInfo, IndexInfo, ForeignKeyInfo, and TriggerInfo. Result sets convert to these objects in AbstractJdbcAgent before JSON serialization, ensuring consistent client-side handling regardless of the underlying database.

Database-Specific Driver Implementations

Each JDBC-compatible database implements a thin subclass of AbstractJdbcAgent within its own module. The three primary implementations demonstrate how drivers adapt the generic framework to database-specific SQL dialects and metadata systems.

Snowflake Agent Implementation

The Snowflake driver resides in agents/drivers/snowflake/src/main/java/com/dbx/agent/snowflake/SnowflakeAgent.java. It configures the Snowflake JDBC driver (net.snowflake.client.jdbc.SnowflakeDriver) and constructs URLs matching the pattern jdbc:snowflake://host:port/?db=database.

For metadata extraction, SnowflakeAgent leverages Snowflake-specific commands:

• SHOW DATABASES and SHOW SCHEMAS for catalog navigation
• INFORMATION_SCHEMA.TABLES for table listings
• GET_DDL() for object source retrieval
• SHOW PRIMARY KEYS for constraint detection

Hive Agent Implementation

Located at agents/drivers/hive/src/main/java/com/dbx/agent/hive/HiveAgent.java, the Hive implementation targets the Apache Hive JDBC driver (org.apache.hive.jdbc.HiveDriver). It builds connection URLs using the Hive2 protocol: jdbc:hive2://host:port/database.

The agent implements Hive-specific metadata queries:

• SHOW DATABASES and SHOW TABLES for catalog operations
• DESCRIBE FORMATTED for detailed table metadata
• USE <schema> via setSchemaSQL() to handle Hive's database context switching

BigQuery Agent Implementation

The BigQuery driver at agents/drivers/bigquery/src/main/java/com/dbx/agent/bigquery/BigQueryAgent.java integrates with Google's BigQuery JDBC driver. It constructs URLs such as jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=....

Metadata extraction relies on BigQuery's INFORMATION_SCHEMA views:

• SELECT * FROM INFORMATION_SCHEMA.SCHEMATA for schemas
• INFORMATION_SCHEMA.TABLES for table listings
• SELECT ddl FROM \project.dataset.TABLES_SUMMARY`` for DDL extraction

JSON-RPC Interaction Flow

The DBX agent service follows a standardized request lifecycle that enables runtime database swapping:

1. Agent Startup – The driver JAR launches via its main method, starting JsonRpcServer to listen on STDIN/STDOUT.
2. Client Request – The DBX client transmits JSON-RPC messages (e.g., listSchemas, executeQuery) over standard streams.
3. Request Dispatch – JsonRpcServer deserializes the request and invokes the corresponding method on the concrete agent.
4. JDBC Execution – The agent opens a JDBC Connection using the driver class from driverClass() and executes database-specific SQL.
5. Result Translation – AbstractJdbcAgent utilities convert ResultSet data into common POJOs (TableInfo, ColumnInfo, etc.).
6. Response Serialization – The server serializes results to JSON and writes to STDOUT.

This flow allows the DBX client to interact uniformly with Snowflake, Hive, BigQuery, or any future JDBC source by simply loading the appropriate agent JAR.

Building and Running Agents

To start a Snowflake agent from the source:

./agents/gradlew :agents:drivers:snowflake:jar

java -cp agents/drivers/snowflake/build/libs/snowflake-agent.jar \
     com.dbx.agent.snowflake.SnowflakeAgent

Once running, the agent accepts JSON-RPC calls. To list schemas:

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "listSchemas",
  "params": {
    "connectParams": {
      "host": "myaccount.snowflakecomputing.com",
      "port": 443,
      "database": "MYDB",
      "user": "alice",
      "password": "*****"
    }
  }
}

Response:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": ["PUBLIC", "ANALYTICS", "RAW_DATA"]
}

To execute a BigQuery query:

{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "executeQuery",
  "params": {
    "connectParams": {
      "host": "bigquery.googleapis.com",
      "port": 443,
      "projectId": "my-project",
      "user": "service-account@example.iam.gserviceaccount.com",
      "password": "*****"
    },
    "sql": "SELECT name, total_bytes FROM `my-project.my_dataset.__TABLES_SUMMARY__` LIMIT 10"
  }
}

To retrieve DDL for a Hive table:

{
  "jsonrpc": "2.0",
  "id": 3,
  "method": "getObjectSource",
  "params": {
    "schema": "default",
    "name": "sales",
    "objectType": "TABLE"
  }
}

Key Source Files

Module	Path	Description
Common Core	agents/common/src/main/java/com/dbx/agent/AbstractJdbcAgent.java	Base class providing generic JDBC handling and the JSON-RPC contract
Common Core	agents/common/src/main/java/com/dbx/agent/JsonRpcServer.java	JSON-RPC server implementation using STDIO
Common Core	agents/common/src/main/java/com/dbx/agent/ConfiguredJdbcAgent.java	Configuration handling (timeouts, SSL) extending the base agent
Snowflake	agents/drivers/snowflake/src/main/java/com/dbx/agent/snowflake/SnowflakeAgent.java	Concrete implementation for Snowflake JDBC
Hive	agents/drivers/hive/src/main/java/com/dbx/agent/hive/HiveAgent.java	Concrete implementation for Apache Hive
BigQuery	agents/drivers/bigquery/src/main/java/com/dbx/agent/bigquery/BigQueryAgent.java	Concrete implementation for Google BigQuery
Build	agents/build.gradle	Gradle configuration for assembling driver JARs
Test Support	agents/test-support/src/main/java/com/dbx/agent/test/JdbcAgentFake.java	In-memory fake implementation for unit testing

Summary

• DBX employs a plug-in architecture comprising a common core (agents/common) and database-specific drivers (agents/drivers/*).
• AbstractJdbcAgent enforces a minimal contract (driver class, URL building, metadata methods) while providing robust default implementations.
• JSON-RPC over STDIO enables language-agnostic client communication and process isolation.
• Database-specific agents implement unique SQL dialects (Snowflake's GET_DDL, Hive's DESCRIBE FORMATTED, BigQuery's INFORMATION_SCHEMA) behind a uniform interface.
• Runtime flexibility allows the DBX client to swap between Snowflake, Hive, BigQuery, and other JDBC sources by loading different agent JARs.

Frequently Asked Questions

How does DBX handle different JDBC driver implementations?

Each database agent implements the driverClass() method to specify its fully-qualified JDBC driver class (e.g., net.snowflake.client.jdbc.SnowflakeDriver for Snowflake). The AbstractJdbcAgent uses this value to load the driver dynamically and establish connections, while concrete agents handle URL construction via buildJdbcUrl(ConnectParams).

What communication protocol does the DBX agent service use?

The service uses JSON-RPC 2.0 transmitted over standard input/output streams. The JsonRpcServer class deserializes incoming requests, invokes the appropriate agent methods, and serializes responses back to JSON, enabling cross-language client compatibility and simple process-based isolation.

How does the BigQuery agent extract table metadata?

The BigQuery implementation queries the INFORMATION_SCHEMA views (specifically SCHEMATA and TABLES) for catalog information. For DDL extraction, it executes SELECT ddl FROM \project.dataset.TABLES_SUMMARY`` to retrieve the native BigQuery table definition, adapting Google's specific metadata schema to the common DBX data model.

Can I extend DBX to support additional JDBC databases?

Yes. You can create a new module under agents/drivers/ containing a class extending AbstractJdbcAgent. Implement the required abstract methods (driverClass(), buildJdbcUrl(), and metadata queries) and provide a main method launching JsonRpcServer. The architecture requires only database-specific SQL logic, as connection handling and JSON-RPC plumbing remain in the common core.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}