# Understanding Magnitude's OpenTelemetry Motel Tracing System: How to Inspect Traces and Spans

> Discover Magnitude's OpenTelemetry Motel tracing system. Learn how to inspect traces and spans locally with this lightweight collector. Get started now.

- Repository: [Magnitude/magnitude](https://github.com/magnitudedev/magnitude)
- Tags: deep-dive
- Published: 2026-09-06

---

**Magnitude's OpenTelemetry Motel is a lightweight, local collector that receives telemetry from ACN and ICN processes via OTLP HTTP/JSON when `MAGNITUDE_OTEL=1` is set, exposing traces and spans through a queryable HTTP API at `http://127.0.0.1:27686`.**

Magnitude ships with **Motel**, a purpose-built OpenTelemetry collector designed for local development and debugging. When enabled, every agent daemon (ACN) and inference server (ICN) instance exports distributed traces to this collector, giving developers granular visibility into RPC calls, span attributes, and log records without external dependencies. This article explains how Motel works internally in the `magnitudedev/magnitude` codebase and demonstrates the exact HTTP endpoints and code patterns for inspecting telemetry data.

## What Is Motel and How It Works

Motel is implemented as a standalone binary that starts automatically with Magnitude's daemon processes. It functions as a minimal **OTLP receiver and storage backend**, accepting spans over HTTP/JSON and persisting them in memory for rapid querying.

**Service Identity and Activation**

According to [`design/inference/observability.md`](https://github.com/magnitudedev/magnitude/blob/main/design/inference/observability.md), telemetry is gated by a single environment variable:

- `MAGNITUDE_OTEL=1` — enables the OTLP exporter in both ACN and ICN processes
- Service name: `magnitude-icn` (used for all inference engine traces)
- Default collector endpoint: `http://127.0.0.1:27686`

This design ensures zero configuration overhead—set one variable and traces flow to the local collector.

## Querying Traces via the Motel HTTP API

The Motel collector exposes a **RESTful HTTP API** documented in [`AGENTS.md`](https://github.com/magnitudedev/magnitude/blob/main/AGENTS.md) (lines 84-100). All endpoints return JSON and support standard HTTP clients.

### Core Endpoints

| Endpoint | Purpose |
|----------|---------|
| `GET /api/health` | Verify collector status |
| `GET /api/services` | List registered services with telemetry |
| `GET /api/traces?service=&lookback=&limit=` | Retrieve recent trace summaries |
| `GET /api/traces/{traceId}` | Fetch complete trace tree with spans |
| `GET /api/spans/{spanId}` | Get single span with attributes and logs |
| `GET /api/logs?service=&lookback=&limit=` | Query recent log records |

### Direct `curl` Examples

Verify the collector is running:

```bash
curl http://127.0.0.1:27686/api/health

```

```json
{ "status": "ok" }

```

List services that have exported telemetry:

```bash
curl http://127.0.0.1:27686/api/services

```

```json
[{ "name": "magnitude-icn", "type": "traces", "version": "..." }]

```

Retrieve recent traces (last 4 hours, max 40):

```bash
curl "http://127.0.0.1:27686/api/traces?service=magnitude-icn&lookback=4h&limit=40"

```

```json
{
  "data": [
    {
      "traceId": "...",
      "rootOperationName": "RpcServer.someMethod",
      "startTime": "...",
      "durationMs": 42
    }
  ]
}

```

Fetch a complete trace tree:

```bash
TRACE_ID=1234567890abcdef
curl "http://127.0.0.1:27686/api/traces/$TRACE_ID"

```

```json
{
  "traceId": "123456...",
  "spans": [
    {
      "spanId": "...",
      "name": "RpcServer.someMethod",
      "parentSpanId": null,
      "attributes": { "rpc.method": "someMethod", "rpc.service": "RpcServer" },
      "events": [...]
    }
  ]
}

```

Inspect a single span with its log records:

```bash
SPAN_ID=abcdef1234567890
curl "http://127.0.0.1:27686/api/spans/$SPAN_ID"

```

```json
{
  "spanId": "abcdef...",
  "attributes": { "key": "value" },
  "logs": [
    { "timestamp": "...", "message": "Processing request", "level": "INFO" }
  ]
}

```

Query recent logs directly:

```bash
curl "http://127.0.0.1:27686/api/logs?service=magnitude-icn&lookback=4h&limit=50"

```

## Programmatic Access via the ACN Dashboard

The **ACN Dashboard** ([`packages/acn-dashboard/src/server.ts`](https://github.com/magnitudedev/magnitude/blob/main/packages/acn-dashboard/src/server.ts)) wraps Motel's HTTP API with TypeScript helpers, handling URL construction and type mapping from raw `MotelTraceSummary` objects to the internal `RpcTraceSummary` format used by the web UI.

In [`packages/acn-dashboard/src/server.ts`](https://github.com/magnitudedev/magnitude/blob/main/packages/acn-dashboard/src/server.ts) (lines 11-13), the collector URL is configured:

```typescript
const MOTEL_URL = process.env.MAGNITUDE_MOTEL_URL || 'http://127.0.0.1:27686';

```

The trace-listing helper (lines 15-26) forwards requests to Motel and transforms responses:

```typescript
import { listRpcTraces } from '@magnitudedev/acn-dashboard/lib/api';

// Enable OTEL before starting any Magnitude processes
process.env.MAGNITUDE_OTEL = '1';

// Fetch RPC-specific traces with typed return values
const traces = await listRpcTraces();
console.log('Recent RPC traces:', traces);

```

This abstraction is useful when building custom dashboards or integrating trace inspection into CI pipelines.

## Architecture and Key Source Files

| File | Responsibility | Key Lines |
|------|---------------|-----------|
| [`AGENTS.md`](https://github.com/magnitudedev/magnitude/blob/main/AGENTS.md) | Motel HTTP API specification | L84-L100 |
| [`design/inference/observability.md`](https://github.com/magnitudedev/magnitude/blob/main/design/inference/observability.md) | OTEL activation and service naming | L18-L27, L20-L27 |
| [`packages/acn-dashboard/src/server.ts`](https://github.com/magnitudedev/magnitude/blob/main/packages/acn-dashboard/src/server.ts) | Dashboard server, Motel client, `listRpcTraces` helper | L11-L13, L15-L26 |

The collector runs **in-process with Magnitude's daemon launcher**, binding to port 27686 on startup. No separate Docker container or external service is required, making it ideal for local development workflows.

## Summary

- **Motel** is Magnitude's built-in OpenTelemetry collector, active when `MAGNITUDE_OTEL=1`
- Default endpoint: `http://127.0.0.1:27686` with six documented HTTP endpoints for health, services, traces, spans, and logs
- The ACN Dashboard ([`packages/acn-dashboard/src/server.ts`](https://github.com/magnitudedev/magnitude/blob/main/packages/acn-dashboard/src/server.ts)) provides typed TypeScript helpers around the raw HTTP API
- Traces from the inference engine are tagged with service name `magnitude-icn` as defined in [`design/inference/observability.md`](https://github.com/magnitudedev/magnitude/blob/main/design/inference/observability.md)
- All telemetry is stored in-memory for fast querying of recent data (typically hours, not days)

## Frequently Asked Questions

### How do I enable tracing in Magnitude?

Set the environment variable `MAGNITUDE_OTEL=1` before starting any ACN or ICN processes. This activates the OTLP HTTP/JSON exporter, which begins sending spans to the Motel collector at `http://127.0.0.1:27686`.

### What service name should I filter for when querying inference traces?

Use `service=magnitude-icn` in your API queries. This is the hardcoded service identifier for all telemetry emitted by Magnitude's inference server, as documented in [`design/inference/observability.md`](https://github.com/magnitudedev/magnitude/blob/main/design/inference/observability.md).

### Can I use Motel in production or CI environments?

Motel is optimized for local development with in-memory storage. For production observability, export traces from Magnitude processes directly to a persistent OpenTelemetry backend (Jaeger, Honeycomb, etc.) using the standard `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable.

### How do I access traces programmatically from my own code?

Either make direct HTTP calls to `http://127.0.0.1:27686/api/traces/*` endpoints, or import the `listRpcTraces` helper from `@magnitudedev/acn-dashboard/lib/api` for a typed interface that handles response parsing automatically.