LiteLLM Observability Integrations: Complete Guide to Langfuse, LangSmith, DataDog, and MLflow

LiteLLM observability integrations automatically capture every LLM request and push structured telemetry to Langfuse, LangSmith, DataDog, and MLflow through a unified, non-blocking pipeline that batches events every 5 seconds.

LiteLLM provides a plug-and-play observability layer that transforms internal execution data into service-specific formats. Each integration follows a standardized six-step pipeline defined in the CustomBatchLogger base class, ensuring high-throughput applications can trace requests without latency penalties or dropped connections.

How LiteLLM Observability Integrations Work

All telemetry flows through a consistent architecture defined in litellm/integrations/custom_batch_logger.py. This design guarantees that Langfuse, LangSmith, DataDog, and MLflow integrations behave identically despite differing underlying APIs.

The Six-Step Logging Pipeline

Every observability integration executes the same sequence:

1. Initialization – The logger class (e.g., LangFuseLogger, DataDogLogger) reads credentials from environment variables or litellm.<service>_params and constructs an HTTP client.
2. Enqueuing – Calls to log_success_event or log_failure_event create a payload object (like LangsmithQueueObject or DatadogPayload) and push it to an in-memory queue.
3. Batching – The CustomBatchLogger accumulates events until either batch_size is reached or flush_interval (default 5 seconds) triggers.
4. Sanitization – Before transmission, truncate_standard_logging_payload_content strips large base-64 blobs and applies optional masking functions to remove secrets.
5. Trace Context Enrichment – If an OpenTelemetry trace is active, _add_trace_context_to_payload injects the trace ID and span ID for end-to-end correlation.
6. Async Transmission – The async_send_batch method posts data via httpx. Errors are caught and logged internally; they never propagate to the user request.

The StandardLoggingPayload Contract

The unified payload format lives in litellm/types/utils.py. Every integration receives a StandardLoggingPayload containing messages, response, metadata, prompt_tokens, and timing data. Each logger only maps these fields to its target schema, eliminating duplicate serialization logic.

Setting Up Langfuse Observability

Langfuse integration supports both direct SDK logging and OpenTelemetry (OTel) traces. Configuration requires only environment variables; no code changes are necessary beyond registering the callback.

import litellm

# .env or shell exports:

# export LANGFUSE_PUBLIC_KEY=pk_...

# export LANGFUSE_SECRET_KEY=sk_...

# export LANGFUSE_HOST=https://cloud.langfuse.com

litellm.callbacks = [litellm.LangFuseOTEL]  # Uses litellm/integrations/langfuse/langfuse_otel.py

In litellm/integrations/langfuse/langfuse.py (lines 10-14), the LangFuseLogger class reads these environment variables during __init__ to authenticate with the Langfuse cloud or self-hosted instance.

Configuring LangSmith Telemetry

The LangSmith integration inherits from CustomBatchLogger and supports request sampling to control log volume.

Sampling Rate Configuration

Set LANGSMITH_SAMPLING_RATE to a float between 0 and 1 to log only a percentage of traffic. The logic resides in _get_sampling_rate_to_use_for_request (lines 503-514 of litellm/integrations/langsmith.py).

import os
import litellm

os.environ["LANGSMITH_API_KEY"] = "ls-..."
os.environ["LANGSMITH_PROJECT"] = "production-routing"
os.environ["LANGSMITH_SAMPLING_RATE"] = "0.1"  # Log 10% of requests

litellm.callbacks = [litellm.LangsmithLogger]  # Defined in litellm/integrations/langsmith.py

Pushing Logs to DataDog

The DataDog logger transmits standardized payloads directly to the DataDog Logs API, bypassing the need for a local Datadog Agent.

import os
import litellm

os.environ["DD_API_KEY"] = "dd_api_key_here"
os.environ["DD_SITE"] = "us5.datadoghq.com"  # Available regions: us1, us3, us5, eu1

litellm.callbacks = [litellm.DataDogLogger]  # Defined in litellm/integrations/datadog/datadog.py

The endpoint URL is constructed in _configure_dd_direct_api (lines 176-182). Each payload includes dynamic tags and, when present, OpenTelemetry trace context injected by _add_trace_context_to_payload.

Recording Runs in MLflow

Unlike other integrations, MLflow requires no LiteLLM-specific environment variables. It respects standard MLflow configuration (MLFLOW_TRACKING_URI, etc.) and creates spans or traces automatically.

import litellm

litellm.callbacks = [litellm.MlflowLogger]  # Defined in litellm/integrations/mlflow.py

The MlflowLogger initializes an MlflowClient in __init__ (line 16) and routes each completion to _start_span_or_trace, mapping the StandardLoggingPayload to MLflow's native run format.

Advanced Payload Customization

All integrations respect the sanitization hooks defined in litellm/integrations/custom_logger.py. You can extend CustomLogger to redact sensitive fields or truncate large content before it reaches external services.

Redacting Sensitive Data

The _strip_base64_from_messages helper (lines 923-960) automatically removes binary image data. You can enforce additional masking by subclassing:

class RedactingLogger(litellm.CustomLogger):
    def log_success_event(self, kwargs, response_obj, start_time, end_time):
        # Truncate and sanitize the standard payload

        safe_payload = self.truncate_standard_logging_payload_content(
            kwargs["standard_logging_object"]
        )
        # Apply custom logic, then forward to base implementation

        return super().log_success_event(kwargs, response_obj, start_time, end_time)

litellm.callbacks = [RedactingLogger()]

Summary

• LiteLLM observability integrations use a unified CustomBatchLogger architecture to push data to Langfuse, LangSmith, DataDog, and MLflow.
• Every request becomes a StandardLoggingPayload that is batched for 5 seconds or until batch_size is hit, then sanitized and transmitted asynchronously.
• Langfuse and LangSmith support sampling and trace context propagation.
• DataDog pushes directly to the REST API with automatic region detection via DD_SITE.
• MLflow requires no additional LiteLLM configuration and creates native runs automatically.
• Failures in telemetry transmission are caught and logged silently, ensuring observability never interrupts LLM inference.

Frequently Asked Questions

How do I enable multiple observability integrations simultaneously?

Assign a list of loggers to litellm.callbacks. Each integration operates independently, so you can route logs to both Langfuse and DataDog concurrently:

litellm.callbacks = [litellm.LangFuseOTEL, litellm.DataDogLogger]

The CustomBatchLogger manages separate queues and flush cycles for each service to prevent cross-contamination of payloads.

What happens if an observability service is temporarily unavailable?

LiteLLM observability integrations are designed to be fail-silent. If async_send_batch encounters a network error or timeout, the exception is caught and logged to the LiteLLM internal logger only. The user request completes normally without backpressure or latency spikes.

Can I filter which requests get logged to LangSmith?

Yes. The LangsmithLogger checks LANGSMITH_SAMPLING_RATE during _get_sampling_rate_to_use_for_request (lines 503-514). For dynamic filtering, subclass CustomLogger and implement conditional logic in log_success_event before calling the parent implementation.

Does LiteLLM support OpenTelemetry tracing for these integrations?

Yes. The DataDog and Langfuse integrations specifically check for active OTel contexts via _add_trace_context_to_payload and _supports_tags. When detected, they inject trace_id and span_id into the outgoing payload, enabling correlation between LiteLLM proxy spans and your application traces.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}