How Feynman Handles Telemetry Using PostHog and OpenTelemetry: A Complete Guide

Feynman implements a dual-layer telemetry architecture that combines lightweight event capture via the PostHog Node.js SDK with OpenTelemetry distributed traces routed to PostHog's AI Observability endpoints.

The advaitpaliwal/feynman repository handles telemetry through two complementary pipelines. The first layer captures discrete CLI events and error metadata using the official posthog-node library, while the second layer emits structured OpenTelemetry traces and logs from the Pi runtime directly to PostHog's AI Observability ingestion endpoints. This design ensures comprehensive observability without exposing sensitive user data such as raw prompts or filesystem paths.

Telemetry Architecture Overview

Feynman's telemetry system separates high-level event tracking from low-level execution tracing. This separation allows the CLI to record user interactions while the underlying Pi runtime captures detailed performance metrics.

PostHog Event Client

The primary event ingestion layer resides in src/telemetry/posthog.ts. This module instantiates a singleton PostHog client configured with a circuit-breaker fetch mechanism to prevent network failures from blocking the main workflow.

// src/telemetry/posthog.ts
export const DEFAULT_POSTHOG_HOST = "https://us.i.posthog.com";

The client exposes two primary functions: captureTelemetryEvent for standard asynchronous event submission and captureTelemetryEventImmediate for latency-critical synchronous reporting.

OpenTelemetry Trace Pipeline

The second layer handles distributed tracing through OpenTelemetry Protocol (OTLP) exporters. The src/pi/runtime.ts file imports getPostHogOtelEnv to generate environment variables that direct trace and log exports to PostHog's specialized AI Observability endpoints at /i/v0/ai/otel and /i/v1/logs respectively.

Configuration and Environment Setup

Telemetry configuration relies on environment variables defined in the project's .env.example file. Users can override the default PostHog host by setting FEYNMAN_POSTHOG_HOST, though the system defaults to https://us.i.posthog.com.

The getPostHogOtelEnv helper function dynamically constructs the OTLP endpoint URLs based on the configured host:

// src/pi/runtime.ts
import { getPostHogOtelEnv } from "../telemetry/posthog.js";

This function returns environment variables pointing to:

• OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: <FEYNMAN_POSTHOG_HOST>/i/v0/ai/otel
• OTEL_EXPORTER_OTLP_LOGS_ENDPOINT: <FEYNMAN_POSTHOG_HOST>/i/v1/logs

Capturing Events with PostHog

The telemetry module provides two distinct patterns for event emission, chosen based on the criticality of the data and the latency constraints of the operation.

Standard Asynchronous Capture

For most CLI interactions, use the captureTelemetryEvent function. This method batches events and transmits them asynchronously using the circuit-breaker protected fetch implementation.

import { captureTelemetryEvent } from "./telemetry/posthog.js";

captureTelemetryEvent("cli.run", {
  command: "feynman rank",
  args: ["--paper", "xyz"],
});

Under the hood, this calls posthogClient.capture() with the configured distinctId and event properties.

Immediate Synchronous Capture

For error reporting or shutdown sequences where data loss is unacceptable, use captureTelemetryEventImmediate. This function awaits the network request before returning, ensuring the event reaches PostHog even if the process exits immediately after.

import { captureTelemetryEventImmediate } from "./telemetry/posthog.js";

await captureTelemetryEventImmediate("error", {
  message: err.message,
  code: err.code,
});

This variant is particularly useful for capturing fatal errors or PaperRank milestone completions that must be recorded before process termination.

OpenTelemetry Integration for AI Observability

Feynman leverages PostHog's AI Observability platform to monitor LLM calls and tool executions within the Pi runtime. This integration requires runtime patching and strict endpoint validation.

Environment Variable Generation

Before spawning the Pi child process, the runtime consolidates OpenTelemetry configuration:

import { getPostHogOtelEnv } from "./telemetry/posthog.js";

process.env = {
  ...process.env,
  ...getPostHogOtelEnv({ host: process.env.FEYNMAN_POSTHOG_HOST }),
};

These variables configure the bundled pi-otel package to export spans to the AI Observability-specific endpoints rather than generic OTLP collectors.

Runtime Patching and Validation

The scripts/lib/pi-otel-patch.mjs module validates that the OTLP exporter points to the expected PostHog URLs. This safety check prevents misconfiguration that could leak trace data to unintended destinations:

if (posthog.url !== "https://us.i.posthog.com/i/v0/ai/otel") {
  throw new Error(`Installed pi-otel changed PostHog trace URL: ${posthog.url}`);
}

Once validated, CLI-level spans populate the posthog.trace_spans table, while Pi-runtime LLM executions appear as $ai_* events queryable through HogQL in the PostHog interface.

Privacy and Safety Mechanisms

Feynman's telemetry implementation enforces strict content policies to protect user privacy. The system deliberately excludes raw prompts, paper contents, filesystem paths, and model generation payloads from all telemetry events.

This sanitization occurs in src/telemetry/posthog.ts and is verified by the test suite in tests/content-policy.test.ts. Additionally, tests/telemetry.test.ts confirms that the circuit-breaker fetch implementation gracefully handles network unreachability without throwing exceptions or blocking CLI operations.

Implementation Examples

Initializing the Telemetry Client

Typically invoked by the CLI entry point in src/cli.ts:

import { initTelemetry } from "./telemetry/posthog.js";

await initTelemetry({
  projectToken: process.env.FEYNMAN_POSTHOG_KEY,
  distinctId: "session-1234",
});

Enabling Full OpenTelemetry Export

To activate distributed tracing for the Pi runtime:

import { getPostHogOtelEnv } from "./telemetry/posthog.js";

process.env = {
  ...process.env,
  ...getPostHogOtelEnv({ host: process.env.FEYNMAN_POSTHOG_HOST }),
};

After setting these variables, the Pi runtime automatically begins exporting spans to the PostHog AI Observability endpoint at /i/v0/ai/otel.

Summary

• Feynman uses a two-layer telemetry system: PostHog events for CLI interactions and OpenTelemetry for runtime traces.
• The posthog-node SDK handles event capture through captureTelemetryEvent and captureTelemetryEventImmediate in src/telemetry/posthog.ts.
• OpenTelemetry traces route to PostHog's AI Observability via OTLP endpoints configured in src/pi/runtime.ts and validated by scripts/lib/pi-otel-patch.mjs.
• A circuit-breaker fetch pattern prevents network failures from impacting CLI performance.
• Strict privacy controls exclude sensitive data like prompts and file paths from all telemetry payloads.

Frequently Asked Questions

How do I disable telemetry in Feynman?

Omit the FEYNMAN_POSTHOG_KEY environment variable during initialization. The initTelemetry function checks for the presence of a project token, and telemetry features remain dormant when this value is undefined.

What is the difference between captureTelemetryEvent and captureTelemetryEventImmediate?

captureTelemetryEvent batches events asynchronously for standard usage, while captureTelemetryEventImmediate awaits the HTTP request before returning. Use the immediate variant only for critical events like fatal errors or process shutdown sequences where data loss is unacceptable.

Where do OpenTelemetry traces appear in PostHog?

CLI-level traces populate the distributed-tracing table posthog.trace_spans, accessible via /i/v1/traces. Pi-runtime LLM and tool execution spans route to the AI Observability endpoint at /i/v0/ai/otel and surface as $ai_* events in the posthog.ai_events table.

Does Feynman collect my prompts or paper content?

No. The telemetry system explicitly excludes raw prompts, paper contents, filesystem paths, and model-generation payloads from all captured events. This content policy is enforced in src/telemetry/posthog.ts and verified by the test suite in tests/content-policy.test.ts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}