Prerequisites for Developing on OpenMetadata: Complete Setup Guide

To develop on OpenMetadata, you need Java 21, Maven 3.9+, Node 20 with Yarn 1.x, Python 3.10 or 3.11, Docker 23+, Git, and GNU Make—plus a verified virtual environment for Python dependencies.

The OpenMetadata platform is a comprehensive, open-source metadata management solution built as a monorepo containing four distinct components: a Java backend (Dropwizard services), a React frontend, a Python ingestion framework, and Dockerized supporting services. Each layer demands specific language runtimes and tooling. This guide walks through every prerequisite for developing on OpenMetadata, verified against the repository's own developer documentation in CLAUDE.md and build configuration files.

Operating System and Core Tools

OpenMetadata's build system assumes a POSIX-compatible environment. All orchestration scripts—whether make targets or docker compose commands—are written for Bash-compatible shells.

Supported Platforms

• Linux: Any recent distribution (Ubuntu 20.04+, RHEL 8+, Debian 11+)
• macOS: macOS 12 (Monterey) or later
• Windows: Windows 10/11 with WSL2 (Windows Subsystem for Linux)

The root README.md explicitly notes that Windows users should use WSL for full compatibility with the build scripts.

Required System Tools

Tool	Minimum Version	Purpose
Git	2.35+	Clone monorepo, manage submodules
GNU Make	4.2+	Central build orchestration

The Makefile at repository root defines the primary developer interface. Every common task—from verifying prerequisites to running tests—is exposed as a make target.

Java Backend Prerequisites

The OpenMetadata backend services are built on Dropwizard, a Java framework for RESTful web services. The entire backend compiles as a multi-module Maven project.

Java 21 JDK

The pom.xml at repository root specifies Java 21 as the compile target. This is non-negotiable—later versions may work but are not explicitly tested, and earlier versions will fail to build.

# Verify Java version

java -version  # Should report 21.x

# Verify JDK (not just JRE)

javac -version  # Should report 21.x

Maven 3.9+

Maven builds the openmetadata-service module and its dependencies. The pom.xml uses modern plugin configurations that require Maven 3.9 or newer.

# Verify Maven version

mvn -version  # Should report 3.9.x or newer, with Java 21

Backend Build Verification

After installing Java and Maven, verify the backend compiles:

# Compile backend without tests, excluding UI module

mvn clean package -DskipTests -DonlyBackend -pl '!openmetadata-ui'

This command, referenced in CLAUDE.md, produces the runnable backend JAR without the time-consuming UI build.

Frontend Prerequisites

The OpenMetadata UI is a React application located in openmetadata-ui/src/main/resources/ui/. The build system uses Yarn 1.x exclusively—npm is not supported.

Node.js 20

The package.json specifies Node 20 as the runtime target. While Node 18 may work, Node 20 is the explicitly tested and recommended version.

# Verify Node version

node -version  # Should report v20.x

Yarn 1.x (Classic)

Critical: OpenMetadata requires Yarn 1.x (the "Classic" line), not Yarn 2/3 with Plug'n'Play. The yarn.lock file and build scripts assume Classic Yarn behavior.

# Verify Yarn version

yarn -version  # Should report 1.22.x or similar 1.x version

Installing Frontend Dependencies

Use the Makefile target for cached installation:

# Install UI dependencies with caching

make yarn_install_cache

Or manually:

cd openmetadata-ui/src/main/resources/ui
yarn install

Running the UI Locally

cd openmetadata-ui/src/main/resources/ui
yarn start

The dev server starts on http://localhost:3000 and proxies API requests to the backend.

Python Ingestion Framework Prerequisites

The ingestion framework is a Python 3.10/3.11 application in the ingestion/ directory. It uses Pydantic 2.x and a large collection of optional connector packages.

Python 3.10 or 3.11

Python 3.11 is recommended for better performance and full compatibility. Python 3.12+ is not yet officially supported due to dependency constraints.

# Verify Python version

python --version  # Should report 3.10.x or 3.11.x

Virtual Environment (Mandatory)

All Python work requires an activated virtual environment. The build scripts and make targets assume env/ exists at repository root.

# Create virtual environment

python -m venv env

# Activate (Linux/macOS)

source env/bin/activate

# Activate (Windows)

env\Scripts\activate

Installing Ingestion Dependencies

The ingestion/Makefile provides the standard installation target:

# Install full development environment with all connector extras

cd ingestion
make install_dev_env

This installs the openmetadata-ingestion package in editable mode with all optional dependencies for connectors (Snowflake, BigQuery, Redshift, etc.).

Python Code Generation

OpenMetadata uses JSON Schema to generate Pydantic models. After any schema change, regenerate:

make generate

This runs the code generators in openmetadata-spec/ and ingestion/.

Docker and Supporting Services

All external dependencies—MySQL, Elasticsearch/OpenSearch, Redis, Kafka—run via Docker Compose.

Docker 23+ and Docker Compose

The docker/development/docker-compose.yml defines the full local stack. Docker Compose v2 (plugin) is required.

# Verify Docker version

docker --version  # 23.x or newer

# Verify Docker Compose

docker compose version  # v2.x

Starting Development Services

# Start all supporting services

docker compose -f docker/development/docker-compose.yml up -d

This brings up:

• MySQL 8.0 (metadata store)
• Elasticsearch 7.x or OpenSearch (search index)
• Redis (caching, task queue)
• Kafka (optional, for event streaming)

Verification Command

After installing all prerequisites, run the comprehensive check:

make prerequisites

This script verifies Java, Maven, Node, Python, Docker, and Yarn versions against the repository's requirements.

Complete Setup Workflow

Follow this sequence to establish a fully functional development environment:

1. Repository Setup

# Clone the monorepo

git clone https://github.com/open-metadata/OpenMetadata.git
cd OpenMetadata

# Verify make works

make help  # Shows available targets

2. Verify Prerequisites

make prerequisites

Address any reported version mismatches before proceeding.

3. Activate Python Environment

python -m venv env
source env/bin/activate  # Linux/macOS

4. Install All Dependencies

# Python ingestion framework

cd ingestion && make install_dev_env && cd ..

# UI dependencies

make yarn_install_cache

# Generate Pydantic models from schemas

make generate

5. Start Docker Services

docker compose -f docker/development/docker-compose.yml up -d

6. Run Backend

mvn clean package -DskipTests -DonlyBackend -pl '!openmetadata-ui'
java -jar openmetadata-service/target/openmetadata-service-*-SNAPSHOT.jar server openmetadata-service/src/main/resources/openmetadata.yaml

7. Run UI (separate terminal)

cd openmetadata-ui/src/main/resources/ui
yarn start

Access the application at http://localhost:3000.

Summary

• Java 21 JDK and Maven 3.9+ are required for the Dropwizard backend—verify with make prerequisites
• Node 20 and Yarn 1.x (Classic) power the React UI—npm is not supported
• Python 3.10 or 3.11 with an activated virtual environment is mandatory for the ingestion framework
• Docker 23+ and Docker Compose run all external services (MySQL, Elasticsearch, Redis, Kafka)
• Use make prerequisites to validate your environment, then follow the sequential setup: Python venv → dependency installation → make generate → Docker services → backend → UI

Frequently Asked Questions

Can I use Python 3.12 for OpenMetadata development?

No, Python 3.12 is not officially supported. The ingestion framework depends on Pydantic 2.x and numerous connector packages that have not been fully validated on Python 3.12. Use Python 3.10 or 3.11 instead, with 3.11 recommended for better performance.

Why does OpenMetadata require Yarn 1.x instead of Yarn 3 or npm?

The UI build system and yarn.lock file in openmetadata-ui/src/main/resources/ui/ assume Yarn Classic (1.x) behavior. Yarn 2/3 with Plug'n'Play changes module resolution in ways that break the existing build scripts. Using npm instead of Yarn is unsupported because the Makefile targets and CI pipelines specifically invoke yarn commands.

What happens if I skip the make prerequisites check?

You can attempt to build directly, but you'll likely encounter cryptic errors from version mismatches. For example, Maven 3.8 may fail to parse the pom.xml due to newer plugin syntax. Node 18 might appear to work but cause subtle React build failures. The make prerequisites script catches these issues before you waste time on failed builds.

Do I need to run all Docker services for basic backend development?

No, but you need at least MySQL and Elasticsearch (or OpenSearch) for the backend to start. The docker/development/docker-compose.yml includes optional services like Kafka for event streaming features. For minimal backend development, you can comment out optional services, though running the full suite ensures you don't encounter missing dependency errors when testing certain features.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}