How to Configure the OpenMetadata Backend for Development

Run docker compose -f docker/development/docker-compose.yml up -d to start MySQL, Elasticsearch, and the OpenMetadata server on the local_app_net Docker network, then customize the behavior using a local .env file.

Setting up a local development environment for the OpenMetadata backend enables you to iterate on Java services and validate API changes without impacting production clusters. This guide shows you how to configure the Dropwizard-based backend using Docker Compose, referencing actual file paths and configuration patterns from the open-metadata/OpenMetadata source repository.

Prerequisites

Before you begin, ensure you have the following tools installed:

• Docker & Docker Compose (version 2.20 or higher)
• Java 21 (the version used by the backend)
• Maven (3.9 or higher)
• make (required for code-generation tasks)

Start the Development Stack

The OpenMetadata backend relies on four containers defined in docker/development/docker-compose.yml:

1. MySQL – the relational store for metadata (port 3306, internal network only)
2. Elasticsearch – the search engine for entity lookup (port 9200)
3. execute-migrate-all – runs Flyway-style native migrations before the server starts
4. OpenMetadata Server – the Dropwizard-based REST service (API on port 8585, admin on 8586)

Run the following command from the repository root:

docker compose -f docker/development/docker-compose.yml up -d

This creates the local_app_net Docker network and exposes the server at http://localhost:8585. The execute-migrate-all container automatically applies pending SQL migrations before the server initializes.

Core Configuration Files

Understanding these key files is essential when you configure the OpenMetadata backend for development:

File	Purpose
docker/development/docker-compose.yml	Declares containers, volumes, health checks, and environment variable mappings for the development stack
conf/openmetadata.yaml	Default server configuration for authentication, pipelines, and logging levels
bootstrap/sql/migrations/native/	Native SQL migration scripts for MySQL and PostgreSQL applied during startup
openmetadata-service/src/main/java/org/openmetadata/service/config/	Java POJOs that map YAML configuration into runtime objects
openmetadata-integration-tests/	Maven module containing the integration test suite that validates API contracts

Customize the Backend Configuration

Override Environment Variables

Create a .env file at the repository root to override defaults. Docker automatically loads this file when starting the stack:

MYSQL_ROOT_PASSWORD=dev-secret
DB_USER=om_dev
DB_USER_PASSWORD=dev-secret
SERVER_PORT=8585
AUTHENTICATION_PROVIDER=basic
AUTHORIZER_CLASS_NAME=org.openmetadata.service.security.DefaultAuthorizer

These variables map directly to the environment block of the openmetadata-server service in the compose file (lines 74–98).

Run Migrations Manually

If you modify native SQL migrations in bootstrap/sql/migrations/native/, re-run the migration container:

docker compose -f docker/development/docker-compose.yml run --rm execute-migrate-all

This container executes bootstrap/openmetadata-ops.sh -d migrate --force, which reads the SERVER_CHANGE_LOG table and applies any pending schemaChanges.sql files.

Configure Authentication (SSO/OIDC)

To switch from basic auth to OIDC, add these variables to your .env file:

AUTHENTICATION_PROVIDER=oidc
OIDC_CLIENT_ID=your-client-id
OIDC_CLIENT_SECRET=your-client-secret
OIDC_DISCOVERY_URI=https://your-idp/.well-known/openid-configuration

The server automatically picks up these settings; see the OIDC section in docker-compose.yml (lines 101–122) for additional provider-specific options.

Verify the Installation

Check the health endpoint to confirm the server is ready:

curl -s http://localhost:8586/healthcheck | jq .

A response of {"status":"healthy"} indicates the backend is running correctly and ready to accept API requests.

Run Integration Tests

Validate your changes using the Maven-based integration test suite:

mvn verify -pl openmetadata-integration-tests -Dskip.unit.tests

This command spins up the Docker stack in the background, executes tests against the API, and tears down the infrastructure automatically.

Connect to the Backend

Java SDK Example

Use the generated Java SDK to interact with the server:

import org.openmetadata.sdk.config.OpenMetadataConfig;
import org.openmetadata.sdk.client.OpenMetadataClient;
import org.openmetadata.sdk.resources.services.DatabaseServiceResource;

public class BackendDemo {
    public static void main(String[] args) throws Exception {
        OpenMetadataConfig cfg = OpenMetadataConfig.builder()
            .host("http://localhost:8585")
            .authProvider("basic")
            .username("admin")
            .password("admin")
            .build();

        OpenMetadataClient.initialize(cfg);
        DatabaseServiceResource ds = new DatabaseServiceResource();
        ds.list(0, 25, null, null).forEach(service -> 
            System.out.println(service.getName()));
    }
}

Python SDK Example

The Python ingestion client reads environment variables from your .env file:

from metadata.sdk import OpenMetadata, OpenMetadataConfig

config = OpenMetadataConfig.from_env()
metadata = OpenMetadata.initialize(config)

for tbl in metadata.list_entities("table"):
    print(tbl.fullyQualifiedName)

cURL Example

Create a database service using the REST API:

curl -X POST http://localhost:8585/api/v1/databaseServices \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $(cat admin.jwt)" \
-d '{
  "name": "dev_mysql",
  "serviceType": "MySQL",
  "connection": {
    "config": {
      "username": "om_dev",
      "password": "dev-secret",
      "hostPort": "mysql:3306"
    }
  }
}'

Retrieve the JWT token from /api/v1/auth/login using credentials defined in conf/openmetadata.yaml.

Summary

• Start the stack using docker compose -f docker/development/docker-compose.yml up -d to launch MySQL, Elasticsearch, and the OpenMetadata server on the local_app_net network
• Customize settings via a .env file to override database credentials, authentication providers, and server ports defined in the compose file
• Run migrations automatically on startup or manually using the execute-migrate-all container with bootstrap/openmetadata-ops.sh -d migrate --force
• Verify health by querying http://localhost:8586/healthcheck for a {"status":"healthy"} response
• Test changes with mvn verify -pl openmetadata-integration-tests to validate API contracts against a temporary Docker stack

Frequently Asked Questions

What ports does the OpenMetadata backend use for development?

The backend exposes port 8585 for the REST API and 8586 for the admin interface and health checks. MySQL runs on 3306 (exposed only to the Docker network), and Elasticsearch uses 9200. These mappings are defined in docker/development/docker-compose.yml.

How do I reset the database during development?

Stop the stack and remove volumes with docker compose -f docker/development/docker-compose.yml down -v, then restart with up -d. This wipes all metadata and starts fresh. Alternatively, to re-apply schema changes without data loss, run the execute-migrate-all container manually.

Can I use PostgreSQL instead of MySQL for local development?

Yes. The migration scripts in bootstrap/sql/migrations/native/ support both MySQL and PostgreSQL. Update DB_DRIVER_CLASS and the connection strings in your .env file, and modify the compose file to use a PostgreSQL image instead of MySQL.

Where are the server logs stored when running via Docker Compose?

Logs are sent to stdout by default and can be viewed with docker logs openmetadata-server. For persistent storage, mount a volume to /logs in the compose file, or check the logging configuration in conf/openmetadata.yaml which controls log rotation and output paths.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}