Monitoring and Logging Options in LLM-App: Beyond MonitoringLevel.NONE

The LLM-App repository supports three monitoring levels—NONE, BASIC, and DETAILED—which control the granularity of runtime metrics, Prometheus exposure, and OpenTelemetry tracing in Pathway pipelines.

The pathwaycom/llm-app templates initialize Pathway computational graphs with zero-overhead defaults, but the underlying Pathway SDK provides comprehensive monitoring and logging options for production observability. By adjusting the monitoring_level parameter in pw.run(), developers can expose everything from basic throughput metrics to per-operator execution statistics.

Understanding MonitoringLevel in the Pathway SDK

The MonitoringLevel enumeration is defined in the Pathway SDK (pathway/common/monitoring.py) and imported throughout the LLM-App templates. It provides three distinct tiers of observability:

MonitoringLevel.NONE

NONE (value 0) disables all runtime metric collection and logging overhead. This is the default setting found in templates like templates/unstructured_to_sql_on_the_fly/app.py and templates/private_rag/app.py, ensuring maximum performance for production workloads where external monitoring is unnecessary.

MonitoringLevel.BASIC

BASIC (value 1) enables essential system and pipeline metrics. When activated, Pathway emits CPU and memory usage, throughput statistics, and end-to-end latency measurements to both the console and a standard Prometheus endpoint. This level strikes a balance between observability and performance for most deployment scenarios.

MonitoringLevel.DETAILED

DETAILED (value 2) provides granular debugging and profiling capabilities. This level includes all BASIC metrics plus per-operator statistics, data-flow graph visualizations, and optional OpenTelemetry distributed tracing. Use this when profiling specific pipeline stages or troubleshooting complex data-flow issues.

Enabling Monitoring in LLM-App Templates

To activate monitoring and logging options beyond the default NONE setting, modify the pw.run() call in your template's entry point.

Enable Basic Monitoring

Replace the default configuration in templates/question_answering_rag/app.py or similar files:

import pathway as pw

# ... define your pipeline ...

pw.run(monitoring_level=pw.MonitoringLevel.BASIC)

This exposes Prometheus metrics on the default port while logging throughput to stdout.

Enable Detailed Monitoring

For debugging templates like templates/drive_alert/app.py, switch to detailed observability:

import pathway as pw

# ... build your computational graph ...

pw.run(monitoring_level=pw.MonitoringLevel.DETAILED)

This configuration captures operator-level execution times and enables OpenTelemetry trace collection for distributed debugging.

Configuring Prometheus Endpoints

When using BASIC or DETAILED levels, you can customize the metrics endpoint via additional pw.run() parameters:

pw.run(
    monitoring_level=pw.MonitoringLevel.BASIC,
    prometheus_port=9090,
    prometheus_address="0.0.0.0"
)

This configuration binds the Prometheus scraper to port 9090 on all interfaces, allowing integration with Grafana or other observability stacks.

Summary

• The LLM-App defaults to MonitoringLevel.NONE in templates like templates/unstructured_to_sql_on_the_fly/app.py for zero-overhead execution.
• MonitoringLevel.BASIC enables CPU, memory, throughput, and latency metrics with Prometheus export capabilities.
• MonitoringLevel.DETAILED adds per-operator statistics, data-flow graphs, and OpenTelemetry tracing for deep debugging.
• Configure custom Prometheus endpoints using prometheus_port and prometheus_address parameters in pw.run().

Frequently Asked Questions

What is the default monitoring level in LLM-App templates?

The default monitoring level is MonitoringLevel.NONE. This is explicitly set in multiple template entry points—including templates/private_rag/app.py and templates/question_answering_rag/app.py—to ensure production deployments run without metric collection overhead.

How do I export metrics to Prometheus from an LLM-App pipeline?

Set monitoring_level=pw.MonitoringLevel.BASIC (or DETAILED) in your pw.run() call, then optionally specify prometheus_port and prometheus_address to configure the scrape endpoint. This exposes standard metrics like throughput and memory usage in Prometheus format.

What additional information does DETAILED monitoring provide compared to BASIC?

DETAILED monitoring includes all BASIC metrics plus per-operator execution statistics, data-flow graph visualizations, and optional OpenTelemetry distributed tracing. This level is implemented in the Pathway SDK to help debug complex computational graphs by showing exactly how data transforms through each pipeline stage.

Can I disable monitoring entirely for maximum performance?

Yes. Use monitoring_level=pw.MonitoringLevel.NONE (the repository default) to disable all runtime metric emission, logging, and trace collection. This configuration eliminates any observability overhead and is recommended for high-throughput production environments where external monitoring is handled by other means.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}