# How to Debug the OpenMetadata Backend: A Complete Developer Guide
> Debug the OpenMetadata backend efficiently. Learn to set log levels, check system health APIs, and use the admin UI to find and fix issues.
- Repository: [OpenMetadata/OpenMetadata](https://github.com/open-metadata/OpenMetadata)
- Tags: how-to-guide
- Published: 2026-04-23
---
**Set `LOG_LEVEL=DEBUG`, check `/api/v1/system/health`, and use the Dropwizard admin health check UI to systematically isolate backend issues in OpenMetadata.**
The OpenMetadata backend is a Dropwizard-based Java service that powers the platform's metadata ingestion, search, and governance capabilities. Learning how to debug the OpenMetadata backend effectively saves hours when troubleshooting startup failures, API errors, or performance bottlenecks. This guide walks through the exact source locations, configuration files, and diagnostic techniques used by the core maintainers.
---
## Understanding the Backend Architecture
Before diving into debugging techniques, you need to understand where the service starts and how components are wired together.
### Entry Point: OpenMetadataApplication
The bootstrap sequence begins in [`OpenMetadataApplication.java`](https://github.com/open-metadata/OpenMetadata/blob/main/OpenMetadataApplication.java), located at [`openmetadata-service/src/main/java/org/openmetadata/service/OpenMetadataApplication.java`](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/java/org/openmetadata/service/OpenMetadataApplication.java). This class extends Dropwizard's `Application` and performs three critical functions:
1. **Registers bundles** — including the cache bundle, search index bundle, and authentication filters
2. **Wires health checks** — `OpenMetadataServerHealthCheck` and cache health checks
3. **Configures JDBI** — database connection pooling and SQL object mapping
Any failure during this startup sequence appears in the logs before the HTTP server begins accepting requests.
### Configuration: OpenMetadataApplicationConfig
Runtime behavior is controlled by [`OpenMetadataApplicationConfig.java`](https://github.com/open-metadata/OpenMetadata/blob/main/OpenMetadataApplicationConfig.java) at [`openmetadata-service/src/main/java/org/openmetadata/service/OpenMetadataApplicationConfig.java`](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/java/org/openmetadata/service/OpenMetadataApplicationConfig.java). This POJO maps directly to your [`openmetadata.yaml`](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata.yaml) file and includes nested configuration for:
- **Database** (`dataSourceFactory`)
- **Search engine** (`elasticSearchConfiguration` or `openSearchConfiguration`)
- **Authentication** (`authenticationConfiguration`)
- **Cache** (`cacheConfiguration`)
When debugging, verify that the loaded configuration matches your expectations by inspecting these values in a debugger or adding temporary log statements.
---
## Debugging Startup Issues
Startup failures are the most common backend problems. Follow this sequence to isolate the root cause.
### Step 1: Enable Debug Logging Before Launch
The logging system is configured in [`logback.xml`](https://github.com/open-metadata/OpenMetadata/blob/main/logback.xml) at [`openmetadata-service/src/main/resources/logback.xml`](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/resources/logback.xml). The root logger uses variable substitution:
```xml
```
Set the environment variable before starting the service:
```bash
export LOG_LEVEL=DEBUG
# or for maximum verbosity
export LOG_LEVEL=TRACE
```
The logs write to `logs/openmetadata-operation.log` and the console. Look for lines containing `DEBUG org.openmetadata.service.OpenMetadataApplication` to confirm the setting took effect.
### Step 2: Verify the Service Started Successfully
After the Dropwizard banner, you should see this log line:
```
INFO org.openmetadata.service.OpenMetadataApplication – Started OpenMetadataApplication
```
If this line is missing, the service failed before completing initialization. Common failure points include:
- **Database connectivity** — Check `dataSourceFactory` configuration and network access
- **Search engine connection** — Elasticsearch or OpenSearch must be reachable
- **Authentication setup** — JWT key configuration errors fail silently in some configurations
### Step 3: Confirm Basic Health with the REST Endpoint
The `SystemResource` class at [`openmetadata-service/src/main/java/org/openmetadata/service/resources/system/SystemResource.java`](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/java/org/openmetadata/service/resources/system/SystemResource.java) exposes a lightweight health check:
```bash
curl -s http://localhost:8585/api/v1/system/health
```
Expected response:
```
OK
```
HTTP 200 confirms the JVM is alive and the HTTP connector is accepting requests. If this fails but the process is running, examine the Dropwizard admin interface next.
---
## Using the Dropwizard Admin Interface
Dropwizard provides a built-in administrative interface on a separate port (default 8585, same as application port in OpenMetadata). This interface exposes critical diagnostic information.
### Health Check Endpoint
Visit `http://localhost:8585/admin/healthcheck` to see all registered health checks:
| Check | Source File | Purpose |
|-------|-------------|---------|
| ServerHealthCheck | [`OpenMetadataServerHealthCheck.java`](https://github.com/open-metadata/OpenMetadata/blob/main/OpenMetadataServerHealthCheck.java) | Basic JVM liveness |
| CacheHealthCheck | [`CacheBundle.java`](https://github.com/open-metadata/OpenMetadata/blob/main/CacheBundle.java) | Redis or in-memory cache connectivity |
The `OpenMetadataServerHealthCheck` at [`openmetadata-service/src/main/java/org/openmetadata/service/OpenMetadataServerHealthCheck.java`](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/java/org/openmetadata/service/OpenMetadataServerHealthCheck.java) is intentionally simple — it returns healthy unless the JVM has crashed. More sophisticated checks like `CacheHealthCheck` in [`CacheBundle.java`](https://github.com/open-metadata/OpenMetadata/blob/main/CacheBundle.java) (at [`openmetadata-service/src/main/java/org/openmetadata/service/cache/CacheBundle.java`](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/java/org/openmetadata/service/cache/CacheBundle.java)) validate actual connectivity to external services.
Red entries indicate component failures. Use these to isolate whether problems are in the database, cache, or search layer.
### Metrics Endpoint
The `/admin/metrics` endpoint exposes JVM memory, thread states, and custom metrics. Look for:
- **JVM heap usage** — Memory pressure causes GC pauses and slowdowns
- **Database connection pool metrics** — `openmetadata.db.pool` gauges show active and idle connections
- **Cache hit/miss rates** — `cache.*` metrics from the cache bundle
### Threads and Diagnostics
Use `/admin/threads` to download a thread dump. This helps identify:
- Deadlocks in database connection pools
- Blocked threads in authentication filters
- Runaway threads in search indexing
---
## Debugging Authentication and Security
Authentication failures are common when configuring SSO or API access. The relevant code lives in the security package.
### JWT Filter Debugging
The `JwtFilter` at [`openmetadata-service/src/main/java/org/openmetadata/service/security/JwtFilter.java`](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/java/org/openmetadata/service/security/JwtFilter.java) validates tokens on every request. Enable debug logging for this package:
```bash
export LOGGING_LEVEL_ORG_OPENMETADATA_SERVICE_SECURITY=DEBUG
```
Or modify [`logback.xml`](https://github.com/open-metadata/OpenMetadata/blob/main/logback.xml) to add:
```xml
```
Debug output shows:
- Token extraction from the `Authorization` header
- Signature validation results
- Claims parsing (user name, email, roles)
### Request Filter Chain
The `ContainerRequestFilterManager` in `OpenMetadataApplication` registers multiple filters. Check the registration order in [`OpenMetadataApplication.java`](https://github.com/open-metadata/OpenMetadata/blob/main/OpenMetadataApplication.java) — filters execute sequentially, and early failures prevent later filters from running.
---
## Debugging Database and JDBI Queries
Database issues manifest as slow queries, connection timeouts, or data inconsistencies.
### Enable SQL Logging
For detailed query visibility, add the JVM flag:
```bash
java -Dorg.jdbi.v3.Logger=DEBUG \
-jar openmetadata-service/target/openmetadata-service-*.jar \
server openmetadata-server.yaml
```
JDBI utilities are in `util/jdbi/` (e.g., `JdbiUtils`). The debug output includes:
- SQL statement text
- Bound parameter values
- Execution timing
### Connection Pool Monitoring
Check [`openmetadata.yaml`](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata.yaml) for the `dataSourceFactory` section. The `maxSize` and `minSize` parameters control pool sizing. Monitor the `/admin/metrics` endpoint for `openmetadata.db.pool.active` and `openmetadata.db.pool.idle` — sustained high active counts indicate pool exhaustion.
---
## Debugging Search Indexing
Search indexing issues cause missing entities or stale search results.
### Search Bundle Logs
The search index bundle logs under `org.openmetadata.service.apps.bundles.searchIndex`. Enable dedicated logging:
```bash
export LOGGING_LEVEL_ORG_OPENMETADATA_SERVICE_APPS_BUNDLES_SEARCHINDEX=DEBUG
```
This logger is defined in [`logback.xml`](https://github.com/open-metadata/OpenMetadata/blob/main/logback.xml) — search for the logger name to confirm the exact package path.
### Reindexing Operations
Forced reindexing can be triggered via API when debugging search inconsistencies. Check `SystemResource` for reindex endpoints — these delegate to the search bundle for execution.
---
## Local Development Debugging
For deepest inspection, run from source with a debugger attached.
### Build and Run from Source
```bash
# Build the service module
mvn clean install -DskipTests
# Run with remote debugging enabled
java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005 \
-jar openmetadata-service/target/openmetadata-service-*.jar \
server openmetadata-server.yaml
```
The [`pom.xml`](https://github.com/open-metadata/OpenMetadata/blob/main/pom.xml) at [`openmetadata-service/pom.xml`](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/pom.xml) defines the build artifact and dependencies.
### IDE Configuration
In IntelliJ IDEA or Eclipse:
1. Create a "Remote JVM Debug" configuration
2. Set host to `localhost` and port to `5005`
3. Set breakpoints in key classes:
- [`OpenMetadataApplication.java`](https://github.com/open-metadata/OpenMetadata/blob/main/OpenMetadataApplication.java) — startup sequence
- [`JwtFilter.java`](https://github.com/open-metadata/OpenMetadata/blob/main/JwtFilter.java) — authentication
- [`SystemResource.java`](https://github.com/open-metadata/OpenMetadata/blob/main/SystemResource.java) — health endpoints
Breakpoints in `OpenMetadataApplication` let you step through bundle registration and catch configuration errors before the server starts.
---
## Summary
- **Enable debug logging** with `LOG_LEVEL=DEBUG` to inspect startup and runtime behavior in `openmetadata-operation.log`
- **Verify basic health** using the `/api/v1/system/health` endpoint and the Dropwizard admin interface at `/admin/healthcheck`
- **Inspect component health checks** including `OpenMetadataServerHealthCheck` and `CacheHealthCheck` to isolate database, cache, or search failures
- **Enable SQL debugging** with `-Dorg.jdbi.v3.Logger=DEBUG` to trace database queries and parameter binding
- **Debug authentication flows** by enabling `org.openmetadata.service.security` logging and inspecting `JwtFilter` behavior
- **Run from source with a remote debugger** attached to set breakpoints in `OpenMetadataApplication`, resource classes, and security filters
---
## Frequently Asked Questions
### How do I check if the OpenMetadata backend is running correctly?
Send a request to `http://localhost:8585/api/v1/system/health` using curl or a browser. A `200 OK` response with body `OK` confirms the JVM is alive and the HTTP connector is accepting requests. For deeper verification, visit `/admin/healthcheck` to see individual component statuses.
### Where are the OpenMetadata backend logs stored?
The service writes operational logs to `logs/openmetadata-operation.log` by default, with console output also enabled. The logging configuration in [`logback.xml`](https://github.com/open-metadata/OpenMetadata/blob/main/logback.xml) at [`openmetadata-service/src/main/resources/logback.xml`](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/resources/logback.xml) controls appenders, log rotation, and the root level via the `LOG_LEVEL` environment variable.
### How do I enable debug logging for database queries in OpenMetadata?
Pass the JVM system property `-Dorg.jdbi.v3.Logger=DEBUG` when starting the service, or set the Logback logger `org.jdbi` to `DEBUG` in [`logback.xml`](https://github.com/open-metadata/OpenMetadata/blob/main/logback.xml). This outputs every SQL statement with bound parameters and execution timing, making it easy to identify slow queries or parameter mismatches.
### What is the best way to debug authentication issues in OpenMetadata?
Enable debug logging for the `org.openmetadata.service.security` package by setting `LOGGING_LEVEL_ORG_OPENMETADATA_SERVICE_SECURITY=DEBUG`. This activates detailed output from `JwtFilter`, showing token extraction, signature validation, and claims parsing. For interactive debugging, attach a remote debugger to `JwtFilter.doFilter()` and step through the validation logic.