How to Monitor different-ai/openwork in Production: Health Endpoints, Custom Monitors, and Metrics

You monitor OpenWork in production by polling its /health endpoint for service status, deploying custom monitor plugins through the plugin system, and collecting JSON-structured logs from all processes.

OpenWork's observability stack is built into its core architecture rather than bolted on. The Den API server exposes a lightweight health endpoint, the plugin system supports first-class monitor components for custom metrics, and all processes emit structured logs suitable for external aggregation. This guide walks through each layer with production-ready code examples.

---

Health Endpoint: The Foundation of OpenWork Monitoring

The primary monitoring interface is the /health route implemented in ee/apps/den-api/src/routes/health.ts. This endpoint returns a standardized JSON payload covering system status, uptime, and component readiness.

Health Response Structure

{
  "status": "ok",
  "uptimeSeconds": 12345,
  "version": "v0.12.3",
  "components": {
    "denApi": "ready",
    "monitorPlugins": "ready"
  }
}

Key fields:

• status – "ok" when all required subsystems are reachable; becomes "degraded" or "error" when dependencies fail
• uptimeSeconds – Process uptime in seconds, critical for detecting unexpected restarts
• version – Running OpenWork version for deployment tracking
• components – Per-component status, including dynamically loaded monitor plugins

The endpoint is automatically registered during Den API server startup with no additional configuration required.

Basic Health Check with cURL

# Poll the health endpoint

curl -s http://localhost:3000/health | jq .

# Extract specific fields for monitoring scripts

curl -s http://localhost:3000/health | jq -r '.status'

---

Configuring Prometheus Scraping for OpenWork

Since the health endpoint returns JSON rather than Prometheus exposition format, use the json_exporter sidecar or a custom recording rule to transform the response.

Prometheus Scrape Configuration

scrape_configs:
  - job_name: 'openwork_health'
    static_configs:
      - targets: ['den-api:3000']
    metrics_path: '/health'
    # Use json_exporter to transform JSON to Prometheus metrics

    
  - job_name: 'openwork_exporters'
    static_configs:
      - targets: ['localhost:7979']  # json_exporter

    relabel_configs:
      - source_labels: [__address__]
        target_label: instance

JSON Exporter Configuration (json_exporter.yml)

metrics:
  - name: openwork_status
    help: OpenWork service status (1=ok, 0=error)
    type: gauge
    path: '{.status}'
    values:
      ok: 1
      degraded: 0.5
      error: 0
      
  - name: openwork_uptime_seconds
    help: Process uptime in seconds
    type: gauge
    path: '{.uptimeSeconds}'

---

Custom Monitor Plugins for Application-Specific Metrics

OpenWork's plugin system recognizes monitor as a first-class component kind. These plugins extend the health endpoint with custom metrics—queue depths, job latencies, database connection pool status, or any application-specific signal.

Plugin Discovery Architecture

The plugin system discovers monitors from organization-specific buckets. The discovery logic resides in ee/apps/den-api/src/routes/org/plugin-system/store.ts, specifically around the knownLeafSegments array that includes monitors as a valid component type.

When a monitor plugin is deployed, the health endpoint automatically incorporates its status into the components object.

Creating a Custom Monitor Plugin

Step 1: Define the plugin manifest at monitors/queueDepth/monitor.json:

{
  "name": "queueDepth",
  "entry": "./monitor.js",
  "type": "monitor",
  "description": "Tracks pending job queue depth"
}

Step 2: Implement the monitor logic at monitors/queueDepth/monitor.js:

// monitors/queueDepth/monitor.js
const { getQueueMetrics } = require('../../lib/queue');

module.exports.check = async () => {
  const metrics = await getQueueMetrics();
  
  return {
    status: metrics.pending < 100 ? 'ready' : 
            metrics.pending < 500 ? 'degraded' : 'error',
    metrics: {
      pending_jobs: metrics.pending,
      processing_jobs: metrics.processing,
      failed_jobs_1h: metrics.recentFailures
    }
  };
};

Step 3: Deploy via the OpenWork CLI:

opencode plugin upload ./monitors/queueDepth --org production

The health endpoint then returns:

{
  "status": "ok",
  "components": {
    "denApi": "ready",
    "monitorPlugins": "ready",
    "queueDepth": "degraded"
  }
}

---

Log Aggregation for Deep Observability

All OpenWork processes emit JSON-structured logs to stdout, designed for ingestion by Fluent Bit, Loki, Datadog Agent, or CloudWatch Logs.

Log Format Structure

{
  "timestamp": "2024-01-15T09:23:47.123Z",
  "level": "info",
  "service": "den-api",
  "message": "Request completed",
  "requestId": "req_abc123",
  "durationMs": 45,
  "path": "/api/v1/jobs"
}

Fluent Bit Configuration Example

[INPUT]
    Name              tail
    Path              /var/log/openwork/*.log
    Parser            json
    Tag               openwork.*

[FILTER]
    Name              modify
    Match             openwork.*
    Add               cluster production-us-east

[OUTPUT]
    Name              loki
    Match             openwork.*
    Host              loki.monitoring.svc
    Port              3100
    Labels            job=openwork,service=$service,level=$level

Electron Client Log Collection

The desktop client exposes its own observability surface. On startup, it prints a banner with the Chrome DevTools Protocol (CDP) URL:

[openwork] dev profile=default cdp=http://127.0.0.1:9223

Capture client logs for debugging:

# Forward client stdout to your log pipeline

pnpm dev 2>&1 | grep -E '"event":"(electron-start|error)"' | \
  jq -c '{t:.timestamp, e:.event, m:.message}'

---

Production Monitoring Architecture

A complete OpenWork monitoring deployment combines these layers:

Layer	Mechanism	Alerting Source
Infrastructure	Node-exporter, cAdvisor	CPU, memory, disk
OpenWork Core	/health endpoint	status != "ok"
Business Logic	Custom monitor plugins	Queue depth, error rates
Client Experience	Electron CDP + logs	Crash rates, load times

Recommended Alerting Rules (Prometheus)

groups:
  - name: openwork
    rules:
      - alert: OpenWorkServiceDown
        expr: openwork_status == 0
        for: 1m
        annotations:
          summary: "OpenWork health check failed"
          
      - alert: OpenWorkHighQueueDepth
        expr: openwork_queue_depth > 500
        for: 5m
        annotations:
          summary: "Job queue backing up"
          
      - alert: OpenWorkFrequentRestarts
        expr: changes(openwork_uptime_seconds[1h]) > 3
        annotations:
          summary: "OpenWork process restarting frequently"

---

Key Source Files for Monitoring

File	Purpose
ee/apps/den-api/src/routes/health.ts	Core health endpoint implementation
ee/apps/den-api/src/routes/org/plugin-system/store.ts	Plugin discovery including monitor components
ee/apps/den-api/src/routes/org/plugin-system/schemas.ts	JSON schema for plugin manifests

---

Summary

• Poll /health on the Den API server for immediate service status—no custom instrumentation required
• Deploy monitor plugins through the plugin system to surface application-specific metrics like queue depths and processing latencies
• Collect JSON logs from all processes using standard log aggregation tools for historical analysis and correlation
• Extend without code changes by uploading monitor plugins via the opencode CLI
• Alert on composite signals combining health status, uptime changes, and custom plugin metrics

---

Frequently Asked Questions

How do I enable the health endpoint?

The health endpoint is automatically enabled when you start the Den API server. No configuration flags are required. The route registers during server initialization in health.ts and listens on the standard HTTP port (default 3000).

Can I expose Prometheus-format metrics directly?

OpenWork does not include a native Prometheus exporter. Use the json_exporter sidecar to transform the /health JSON response, or create a monitor plugin that exposes a separate /metrics endpoint with Prometheus exposition format. The plugin system in store.ts supports this pattern.

What happens when a monitor plugin fails?

If a monitor plugin's check() function throws or returns an error status, the health endpoint marks that component as error in the components object. The overall status becomes "degraded" or "error" depending on whether the plugin is marked as required. Check server logs for stack traces.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}