How Real-Time Log Streaming and Container Metrics Work in Openship

Openship bridges Docker Engine API streams with WebSocket broadcasting to deliver live log lines and resource statistics to web dashboards, desktop apps, and CLI clients.

Openship (the oblien/openship repository) implements a decoupled, event-driven architecture for real-time log streaming and container metrics. The system wraps Docker's native HTTP API, transforms container output into typed events, and distributes them via Socket.IO to multiple consumer interfaces without tight coupling between the data sources and UI layers.

Docker Engine API Integration

The monitoring pipeline starts in the adapters package, where a thin TypeScript wrapper around Docker's HTTP API handles low-level stream attachments.

Log Stream Acquisition

In packages/adapters/src/docker/docker-client.ts, the client initiates long-running HTTP requests to Docker's log endpoint:

GET /containers/:id/logs?stdout=1&stderr=1&follow=1

The follow=1 parameter keeps the connection open, returning a Node.js stream that emits newline-delimited log chunks as bytes. This raw stream is wrapped and returned to callers as a consumable interface without parsing logic.

Live Metrics Collection

Resource statistics are captured through a parallel endpoint:

GET /containers/:id/stats?stream=1

This returns JSON-encoded stats objects containing CPU, memory, I/O, and network metrics. The adapter surfaces these as a readable stream that yields complete JSON blobs, which higher layers parse into structured data.

Server-Side Event Processing

Once the Docker streams are active, the core package transforms transport-level data into domain events.

The Log-Stream Service

Located at packages/core/src/monitoring/log-stream.ts, this service creates a PassThrough stream for each monitored container. It registers listeners on both the log and stats streams from the Docker client, then wraps incoming data in a standardized envelope:

{
  type: 'log' | 'metrics',
  payload: string | object,
  containerId: string
}

When new data arrives, the service emits these envelopes on a private EventEmitter. This abstraction decouples Docker API specifics from transport concerns, allowing the same event stream to feed multiple output channels.

WebSocket Broadcasting

The packages/core/src/socket.ts module initializes a Socket.IO server that hooks into the log-stream service's EventEmitter. When an envelope is emitted, the socket server broadcasts it to all connected clients that have explicitly subscribed to that container ID.

Clients initiate monitoring by emitting a subscribe message with their target container ID. The server maintains a registry of socket-to-container mappings, ensuring bandwidth is conserved by routing Docker streams only to subscribed sessions.

Client-Side Implementation

Both the Electron desktop app and the web dashboard consume the same Socket.IO events through shared React components.

React Log Viewer

The LogViewer.tsx component in apps/dashboard/src/components/ establishes a socket connection, dispatches the subscribe event, and appends each incoming log type message to a scrollable <pre> element:

useEffect(() => {
  const socket = io();
  socket.emit('subscribe', { containerId });

  socket.on('message', (msg) => {
    if (msg.type === 'log' && msg.containerId === containerId) {
      setLines((prev) => [...prev, msg.payload]);
    }
  });

  return () => socket.disconnect();
}, [containerId]);

The component automatically scrolls to the newest entry, presenting a terminal-like experience for watching container output in real time.

Real-Time Metrics Charts

ContainerMetrics.tsx listens for metrics type messages and parses the JSON payload to extract cpu_percent and memory_usage values:

socket.on('message', (msg) => {
  if (msg.type === 'metrics' && msg.containerId === containerId) {
    const { cpu_percent, memory_usage } = msg.payload;
    setData((prev) => ({
      cpu: [...prev.cpu, cpu_percent],
      mem: [...prev.mem, memory_usage / (1024 * 1024)],
    }));
  }
});

These values feed into a lightweight chart library to render live CPU percentage and memory consumption (converted to MiB) as time-series visualizations.

CLI Consumption

The CLI command at packages/cli/src/commands/logs.ts runs the same log-stream service in "foreground" mode. When invoked with openship logs <service>, it creates a temporary socket client that prints log lines to stdout. With the --metrics flag, the CLI draws a simple ASCII graph of resource statistics, reusing the same event infrastructure as the graphical interfaces.

Practical Implementation Example

To attach to a container's logs and metrics programmatically:

import { dockerClient } from '@/adapters/docker/docker-client';
import { EventEmitter } from 'events';
import { socket } from '@/core/socket';

const logEmitter = new EventEmitter();

export function attachContainerLogs(containerId: string) {
  const logStream = dockerClient.getContainerLogs(containerId);
  const statsStream = dockerClient.getContainerStats(containerId);

  logStream.on('data', (chunk) => {
    logEmitter.emit('message', {
      type: 'log',
      payload: chunk.toString('utf8'),
      containerId,
    });
  });

  statsStream.on('data', (chunk) => {
    logEmitter.emit('message', {
      type: 'metrics',
      payload: JSON.parse(chunk.toString('utf8')),
      containerId,
    });
  });
}

// Broadcast to all WebSocket clients that subscribed to this container
logEmitter.on('message', (msg) => socket.broadcast(msg));

This pattern ensures the Docker wrapper knows nothing about sockets, while the broadcasting layer remains agnostic to the Docker API's specific response formats.

Summary

• Docker API wrapper (docker-client.ts): Handles GET /containers/:id/logs and GET /containers/:id/stats with streaming parameters, exposing Node.js streams to the application layer.
• Log-stream service (log-stream.ts): Converts raw Docker streams into typed events using PassThrough streams and an EventEmitter, decoupling data acquisition from distribution.
• WebSocket transport (socket.ts): Uses Socket.IO to route events only to subscribed clients based on container ID, minimizing network overhead.
• Multi-interface consumers: React components (LogViewer.tsx, ContainerMetrics.tsx) and the CLI (logs.ts) share the same event contract, enabling consistent real-time monitoring across web, desktop, and terminal environments.

Frequently Asked Questions

How does Openship handle large log volumes without overwhelming the UI?

The server-side log-stream.ts service implements backpressure handling through Node.js PassThrough streams, which naturally respect flow control. Additionally, the Socket.IO layer only broadcasts to clients that have explicitly subscribed to a specific container ID, preventing unrelated sessions from receiving data they haven't requested.

Can I consume container metrics from the CLI without opening the dashboard?

Yes. The packages/cli/src/commands/logs.ts module supports a --metrics flag that connects to the same Socket.IO backend used by the web interface. It renders an ASCII graph of CPU and memory statistics directly in the terminal, utilizing identical event streams and subscription logic as the React components.

What format does Openship use for real-time metrics data?

Metrics messages follow a JSON envelope structure with type: 'metrics', containing a payload parsed from Docker's stats API. The payload includes fields such as cpu_percent and memory_usage (in bytes), which the UI converts to MiB before rendering in charts.

Is the WebSocket connection secure for remote deployments?

The Socket.IO server in packages/core/src/socket.ts supports standard Node.js TLS configuration. As implemented in the oblien/openship source, production deployments should terminate TLS at a reverse proxy or configure the Socket.IO server with valid certificates, following standard practices for securing WebSocket transports.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}