How to Configure Logging and Traceability in Neuro SAN: Complete Setup Guide

Neuro SAN provides a three-layer observability stack combining HOCON-based structured logging, a subprocess log bridge with Rich formatting, and optional OpenTelemetry tracing via the Phoenix plugin.

The cognizant-ai-lab/neuro-san-studio repository implements a flexible logging pipeline designed for AI agent workflows. To configure logging and traceability in Neuro SAN, you manipulate three core components: the HOCON configuration file (logging.hocon), the ProcessLogBridge runtime handler, and the optional PhoenixPlugin for distributed tracing. Each layer captures different aspects of system behavior, from structured JSON logs to end-to-end request tracing across subprocess boundaries.

Understanding the Three-Layer Architecture

According to the source code in cognizant-ai-lab/neuro-san-studio, observability splits into Configuration, Runtime Bridge, and Tracing layers.

Configuration Layer (logging.hocon): Stores the Python logging.config.dictConfig dictionary that defines formatters, filters, and handlers. This file controls the root logger and module-specific log levels.

Runtime Bridge (plugins/log_bridge/process_log_bridge.py): Captures raw stdout and stderr from subprocesses (such as model servers or external tools), enriches messages with structured metadata, and formats output using the Rich library. The ProcessLogBridge class handles log rotation, JSON parsing, and pretty-printing.

Tracing Layer (plugins/phoenix/phoenix_plugin.py): Manages OpenTelemetry instrumentation via the PhoenixPlugin class. When enabled, this layer instruments AI SDKs (OpenAI, LangChain, LiteLLM, Anthropic, MCP) and exports spans to an OTLP collector.

Configuring Structured Logging with HOCON

The default logging behavior resides in logging.hocon at the repository root. This file follows the standard Python dictConfig schema with Neuro SAN-specific extensions.

Key Components:

• formatters → structured: Emits single-line JSON containing message, user_id, Timestamp, source, message_type, and request_id fields.
• filters → inject_context: Applies neuro_san.service.http.logging.log_context_filter.LogContextFilter to inject request-level metadata automatically.
• handlers: The console handler streams color-rich output to sys.stdout, while http_console adds the inject_context filter for HTTP server contexts. The null handler discards noisy library logs.

Override the default configuration by setting the PYTHON_LOGGING_CONFIG environment variable or passing --logging-config to run.py:

# Create a custom configuration that enables DEBUG for your module

cat > my_logging.json <<EOF
{
  "loggers": {
    "my_module": {
      "level": "DEBUG",
      "handlers": ["console"],
      "propagate": false
    }
  }
}
EOF

# Apply the configuration

PYTHON_LOGGING_CONFIG=my_logging.json python -m neuro_san.run

Bridging Subprocess Logs with ProcessLogBridge

When Neuro SAN spawns subprocesses, the ProcessLogBridge class in plugins/log_bridge/process_log_bridge.py intercepts their output to ensure consistent formatting. The bridge creates a Rich console with configurable themes and optionally attaches a timed rotating file handler when you specify a runner_log_file.

Processing Pipeline:

1. Capture: Reads stdout/stderr lines from attached processes.
2. Parse: Attempts strict JSON parsing, falls back to tolerant fragment parsing, then plain-text handling.
3. Enrich: Infers log level from message_type fields or textual severity keywords (DEBUG, INFO, WARN).
4. Format: Reassembles multiline JSON blocks and pretty-prints tracebacks using Rich syntax highlighting.

Use ProcessLogBridge directly in custom scripts:

import subprocess
from plugins.log_bridge.process_log_bridge import ProcessLogBridge

# Start a subprocess

proc = subprocess.Popen(
    ["python", "-c", "print('{\"message\":\"agent started\",\"request_id\":\"abc123\"}')"],
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    text=True,
)

# Bridge output to console and rotating file

bridge = ProcessLogBridge(level="INFO", runner_log_file="logs/neuro_san.log")
bridge.attach_process_logger(proc, "agent_process", "logs/agent.log")
proc.wait()

Enabling OpenTelemetry Tracing with Phoenix

Distributed tracing is opt-in via the PhoenixPlugin class in plugins/phoenix/phoenix_plugin.py. Set PHOENIX_ENABLED=true to activate the tracer provider during server startup in servers/neuro_san/neuro_san_server_wrapper.py.

Environment Configuration:

• PHOENIX_ENABLED: Master switch (false by default).
• PHOENIX_AUTOSTART: When true, the plugin spawns the Phoenix UI collector as a subprocess and waits for the port to become reachable.
• OTEL_SERVICE_NAME: Logical identifier (neuro-san-demos by default).
• OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: Collector URL (http://localhost:6006/v1/traces by default).
• PHOENIX_OTEL_REGISTER: Prefer the one-step phoenix.otel.register() auto-instrumentation path (true by default).

The plugin attempts phoenix.otel.register() first; if the package is missing, it falls back to a manual tracer provider with an OTLP HTTP exporter. Instrumentation for OpenAI, LangChain, LiteLLM, Anthropic, and MCP occurs lazily via the openinference library.

Start the server with full tracing:

export PHOENIX_ENABLED=true
export PHOENIX_AUTOSTART=true
export OTEL_SERVICE_NAME=production-agent
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:6006/v1/traces

python -m neuro_san.run

Integrating Tracing into Application Code

Once PhoenixPlugin initializes, all supported SDK calls emit spans automatically. Initialize the plugin explicitly in custom scripts to ensure tracing context propagates:

from plugins.phoenix.phoenix_plugin import PhoenixPlugin
import openai

# Initialize tracing (no-op if PHOENIX_ENABLED=false)

PhoenixPlugin().initialize()

# This call generates OpenTelemetry spans automatically

response = openai.ChatCompletion.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Analyze this data"}]
)

Summary

• Use logging.hocon to define structured JSON formatters and context injection filters that automatically tag logs with request_id and user_id.
• Set PYTHON_LOGGING_CONFIG or pass --logging-config to override default logging behavior without modifying source code.
• Leverage ProcessLogBridge to capture subprocess output with automatic JSON parsing, log level inference, and Rich console formatting.
• Enable PHOENIX_ENABLED=true to activate OpenTelemetry tracing with auto-instrumentation for major AI SDKs, optionally auto-starting the Phoenix UI collector via PHOENIX_AUTOSTART.

Frequently Asked Questions

How do I redirect logs to a file instead of stdout?

Use the runner_log_file parameter when instantiating ProcessLogBridge in plugins/log_bridge/process_log_bridge.py. This attaches a timed rotating file handler that persists logs across restarts while still printing color-coded output to the console.

Can Neuro SAN parse plain-text logs from external tools?

Yes. The ProcessLogBridge implementation attempts strict JSON parsing first, then falls back to a tolerant fragment parser, and finally handles plain text. It infers severity levels from textual keywords like DEBUG or ERROR when JSON metadata is unavailable.

How do I disable tracing without code changes?

Set the environment variable PHOENIX_ENABLED=false (the default). When disabled, PhoenixPlugin.initialize() returns immediately without configuring a tracer provider, making it a safe no-op for production environments where tracing infrastructure is unavailable.

Which handler should I use for the HTTP server context?

Use the http_console handler defined in logging.hocon. Unlike the standard console handler, http_console attaches the inject_context filter from neuro_san.service.http.logging.log_context_filter.LogContextFilter, ensuring every log entry includes request-specific metadata like user_id and request_id.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}