# How to Configure Logging and Traceability in Neuro SAN: Complete Setup Guide

> Master Neuro SAN logging and traceability setup. This guide details its three-layer observability stack including HOCON logging, Rich formatting, and OpenTelemetry tracing for enhanced insights.

- Repository: [Cognizant AI Lab/neuro-san-studio](https://github.com/cognizant-ai-lab/neuro-san-studio)
- Tags: how-to-guide
- Published: 2026-02-27

---

**Neuro SAN provides a three-layer observability stack combining HOCON-based structured logging, a subprocess log bridge with Rich formatting, and optional OpenTelemetry tracing via the Phoenix plugin.**

The cognizant-ai-lab/neuro-san-studio repository implements a flexible logging pipeline designed for AI agent workflows. To configure logging and traceability in Neuro SAN, you manipulate three core components: the **HOCON configuration file** (`logging.hocon`), the **ProcessLogBridge** runtime handler, and the optional **PhoenixPlugin** for distributed tracing. Each layer captures different aspects of system behavior, from structured JSON logs to end-to-end request tracing across subprocess boundaries.

## Understanding the Three-Layer Architecture

According to the source code in `cognizant-ai-lab/neuro-san-studio`, observability splits into **Configuration**, **Runtime Bridge**, and **Tracing** layers.

**Configuration Layer** (`logging.hocon`): Stores the Python `logging.config.dictConfig` dictionary that defines formatters, filters, and handlers. This file controls the root logger and module-specific log levels.

**Runtime Bridge** ([`plugins/log_bridge/process_log_bridge.py`](https://github.com/cognizant-ai-lab/neuro-san-studio/blob/main/plugins/log_bridge/process_log_bridge.py)): Captures raw stdout and stderr from subprocesses (such as model servers or external tools), enriches messages with structured metadata, and formats output using the Rich library. The `ProcessLogBridge` class handles log rotation, JSON parsing, and pretty-printing.

**Tracing Layer** ([`plugins/phoenix/phoenix_plugin.py`](https://github.com/cognizant-ai-lab/neuro-san-studio/blob/main/plugins/phoenix/phoenix_plugin.py)): Manages OpenTelemetry instrumentation via the `PhoenixPlugin` class. When enabled, this layer instruments AI SDKs (OpenAI, LangChain, LiteLLM, Anthropic, MCP) and exports spans to an OTLP collector.

## Configuring Structured Logging with HOCON

The default logging behavior resides in `logging.hocon` at the repository root. This file follows the standard Python `dictConfig` schema with Neuro SAN-specific extensions.

**Key Components:**

- **`formatters` → `structured`**: Emits single-line JSON containing `message`, `user_id`, `Timestamp`, `source`, `message_type`, and `request_id` fields.
- **`filters` → `inject_context`**: Applies `neuro_san.service.http.logging.log_context_filter.LogContextFilter` to inject request-level metadata automatically.
- **`handlers`**: The `console` handler streams color-rich output to `sys.stdout`, while `http_console` adds the `inject_context` filter for HTTP server contexts. The `null` handler discards noisy library logs.

Override the default configuration by setting the `PYTHON_LOGGING_CONFIG` environment variable or passing `--logging-config` to [`run.py`](https://github.com/cognizant-ai-lab/neuro-san-studio/blob/main/run.py):

```bash

# Create a custom configuration that enables DEBUG for your module

cat > my_logging.json <<EOF
{
  "loggers": {
    "my_module": {
      "level": "DEBUG",
      "handlers": ["console"],
      "propagate": false
    }
  }
}
EOF

# Apply the configuration

PYTHON_LOGGING_CONFIG=my_logging.json python -m neuro_san.run

```

## Bridging Subprocess Logs with ProcessLogBridge

When Neuro SAN spawns subprocesses, the `ProcessLogBridge` class in [`plugins/log_bridge/process_log_bridge.py`](https://github.com/cognizant-ai-lab/neuro-san-studio/blob/main/plugins/log_bridge/process_log_bridge.py) intercepts their output to ensure consistent formatting. The bridge creates a **Rich console** with configurable themes and optionally attaches a **timed rotating file handler** when you specify a `runner_log_file`.

**Processing Pipeline:**

1. **Capture**: Reads stdout/stderr lines from attached processes.
2. **Parse**: Attempts strict JSON parsing, falls back to tolerant fragment parsing, then plain-text handling.
3. **Enrich**: Infers log level from `message_type` fields or textual severity keywords (`DEBUG`, `INFO`, `WARN`).
4. **Format**: Reassembles multiline JSON blocks and pretty-prints tracebacks using Rich syntax highlighting.

Use `ProcessLogBridge` directly in custom scripts:

```python
import subprocess
from plugins.log_bridge.process_log_bridge import ProcessLogBridge

# Start a subprocess

proc = subprocess.Popen(
    ["python", "-c", "print('{\"message\":\"agent started\",\"request_id\":\"abc123\"}')"],
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    text=True,
)

# Bridge output to console and rotating file

bridge = ProcessLogBridge(level="INFO", runner_log_file="logs/neuro_san.log")
bridge.attach_process_logger(proc, "agent_process", "logs/agent.log")
proc.wait()

```

## Enabling OpenTelemetry Tracing with Phoenix

Distributed tracing is opt-in via the `PhoenixPlugin` class in [`plugins/phoenix/phoenix_plugin.py`](https://github.com/cognizant-ai-lab/neuro-san-studio/blob/main/plugins/phoenix/phoenix_plugin.py). Set `PHOENIX_ENABLED=true` to activate the tracer provider during server startup in [`servers/neuro_san/neuro_san_server_wrapper.py`](https://github.com/cognizant-ai-lab/neuro-san-studio/blob/main/servers/neuro_san/neuro_san_server_wrapper.py).

**Environment Configuration:**

- **`PHOENIX_ENABLED`**: Master switch (`false` by default).
- **`PHOENIX_AUTOSTART`**: When `true`, the plugin spawns the Phoenix UI collector as a subprocess and waits for the port to become reachable.
- **`OTEL_SERVICE_NAME`**: Logical identifier (`neuro-san-demos` by default).
- **`OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`**: Collector URL (`http://localhost:6006/v1/traces` by default).
- **`PHOENIX_OTEL_REGISTER`**: Prefer the one-step `phoenix.otel.register()` auto-instrumentation path (`true` by default).

The plugin attempts `phoenix.otel.register()` first; if the package is missing, it falls back to a manual tracer provider with an OTLP HTTP exporter. Instrumentation for OpenAI, LangChain, LiteLLM, Anthropic, and MCP occurs lazily via the `openinference` library.

Start the server with full tracing:

```bash
export PHOENIX_ENABLED=true
export PHOENIX_AUTOSTART=true
export OTEL_SERVICE_NAME=production-agent
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:6006/v1/traces

python -m neuro_san.run

```

## Integrating Tracing into Application Code

Once `PhoenixPlugin` initializes, all supported SDK calls emit spans automatically. Initialize the plugin explicitly in custom scripts to ensure tracing context propagates:

```python
from plugins.phoenix.phoenix_plugin import PhoenixPlugin
import openai

# Initialize tracing (no-op if PHOENIX_ENABLED=false)

PhoenixPlugin().initialize()

# This call generates OpenTelemetry spans automatically

response = openai.ChatCompletion.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Analyze this data"}]
)

```

## Summary

- **Use `logging.hocon`** to define structured JSON formatters and context injection filters that automatically tag logs with `request_id` and `user_id`.
- **Set `PYTHON_LOGGING_CONFIG`** or pass `--logging-config` to override default logging behavior without modifying source code.
- **Leverage `ProcessLogBridge`** to capture subprocess output with automatic JSON parsing, log level inference, and Rich console formatting.
- **Enable `PHOENIX_ENABLED=true`** to activate OpenTelemetry tracing with auto-instrumentation for major AI SDKs, optionally auto-starting the Phoenix UI collector via `PHOENIX_AUTOSTART`.

## Frequently Asked Questions

### How do I redirect logs to a file instead of stdout?

Use the `runner_log_file` parameter when instantiating `ProcessLogBridge` in [`plugins/log_bridge/process_log_bridge.py`](https://github.com/cognizant-ai-lab/neuro-san-studio/blob/main/plugins/log_bridge/process_log_bridge.py). This attaches a timed rotating file handler that persists logs across restarts while still printing color-coded output to the console.

### Can Neuro SAN parse plain-text logs from external tools?

Yes. The `ProcessLogBridge` implementation attempts strict JSON parsing first, then falls back to a tolerant fragment parser, and finally handles plain text. It infers severity levels from textual keywords like `DEBUG` or `ERROR` when JSON metadata is unavailable.

### How do I disable tracing without code changes?

Set the environment variable `PHOENIX_ENABLED=false` (the default). When disabled, `PhoenixPlugin.initialize()` returns immediately without configuring a tracer provider, making it a safe no-op for production environments where tracing infrastructure is unavailable.

### Which handler should I use for the HTTP server context?

Use the `http_console` handler defined in `logging.hocon`. Unlike the standard `console` handler, `http_console` attaches the `inject_context` filter from `neuro_san.service.http.logging.log_context_filter.LogContextFilter`, ensuring every log entry includes request-specific metadata like `user_id` and `request_id`.