How to Configure ADR Sensor Output Formats: JSON vs JSONL

Configure the ADR Sensor's output format using the --output-format flag with values "json" (default) or "jsonl" to control whether events are written as a single JSON document or as line-delimited JSON records.

The ADR Sensor from Uber's ADR repository supports two serialization formats for its collected telemetry data. This guide explains how the output_format parameter flows from the command-line interface through to the AgentObserver.save_to_file method, with complete code examples showing both CLI and programmatic usage.

Where the Output Format Is Defined

The --output-format argument is registered in Sensor/adr_sensor/cli.py as a restricted-choice flag with a default of "json":

# From Sensor/adr_sensor/cli.py, lines 61-65

parser.add_argument(
    '--output-format',
    choices=['json', 'jsonl'],
    default='json',
    help='Output format: "json" for a single JSON file, "jsonl" for line-delimited JSON'
)

This restriction ensures only valid format strings reach the observer logic. The main() function then passes this value directly to the observer:

# From Sensor/adr_sensor/cli.py, lines 141-146

observer.save_to_file(
    entries,
    system_cfg,
    output_format=args.output_format,
    output_dir=args.output_dir
)

How the Observer Handles Each Format

The AgentObserver.save_to_file method in Sensor/adr_sensor/observer.py implements distinct serialization logic based on the output_format parameter:

JSON Format (Default)

When output_format="json", the observer writes:

• Events file: agent_event_logs_<timestamp>.json containing a single JSON object with an "entries" array
• System config file: system_configuration_<timestamp>.json as a single JSON document

From Sensor/adr_sensor/observer.py, lines 84-92:

# JSON branch: single file with nested structure

output_path = output_dir / f"agent_event_logs_{timestamp}.json"
with open(output_path, 'w') as f:
    json.dump({"entries": [entry.to_dict() for entry in entries]}, f, indent=2)

JSONL Format (Line-Delimited)

When output_format="jsonl", the observer writes compact, newline-separated records:

From Sensor/adr_sensor/observer.py, lines 94-98:

# JSONL branch: one event per line

output_path = output_dir / f"agent_event_logs_{timestamp}.jsonl"
with open(output_path, 'w') as f:
    for entry in entries:
        f.write(json.dumps(entry.to_dict(), separators=(',', ':')) + '\n')

The same pattern applies to system configuration output (lines 102-116), ensuring consistency across all generated files.

CLI Usage Examples

Default JSON Output

# Single JSON document with all events nested under "entries"

adr-sensor --output-dir ./telemetry-data

Explicit JSONL Output

# One compact JSON object per line—ideal for streaming pipelines

adr-sensor --output-format jsonl --output-dir ./telemetry-data

Programmatic Configuration

Import AgentObserver directly to configure output format in Python:

from pathlib import Path
from adr_sensor.observer import AgentObserver

# Initialize observer

observer = AgentObserver(output_dir=Path("./output"))

# Ingest data from all configured sources

entries, system_cfg = observer.ingest_all(source="all")

# Save as JSONL for log aggregation systems

observer.save_to_file(
    entries,
    system_cfg,
    output_format="jsonl",      # Use "json" for single-document output

    output_dir=Path("./output")
)

Output File Comparison

Format	Events Filename	Structure	Best For
JSON	agent_event_logs_20241012_101530.json	Single object with "entries": [...]	Human inspection, small datasets
JSONL	agent_event_logs_20241012_101530.jsonl	One {"...}\n{...} per line	Streaming pipelines, jq processing, big data tools

Both formats receive identical AgentEvent data from the parsers in Sensor/adr_sensor/parsers/, serialized according to schemas in Sensor/adr_sensor/schemas/.

Summary

• The --output-format flag in cli.py controls both events and system configuration output
• "json" produces readable, nested documents suitable for manual analysis
• "jsonl" produces streamable, line-delimited records optimized for programmatic consumption
• The save_to_file method in observer.py implements format-specific serialization with appropriate filename extensions

Frequently Asked Questions

What is the default output format if I don't specify --output-format?

The ADR Sensor defaults to "json" as defined in Sensor/adr_sensor/cli.py line 65. This produces a single JSON file with an "entries" array containing all collected events.

Does JSONL format change the underlying data schema?

No. According to Sensor/adr_sensor/observer.py, both formats serialize the same AgentEvent.to_dict() output. JSONL simply writes each event as an independent compact JSON object on its own line rather than nesting all events in a parent array.

Can I use different formats for events and system configuration?

No. The current implementation in Sensor/adr_sensor/observer.py applies the same output_format value to both files. Lines 84-116 show identical branching logic for events and system configuration using the single parameter passed from main().

Which format should I choose for large-scale log processing?

Choose JSONL. The line-delimited format allows streaming parsers to process events one at a time without loading the entire file into memory, and it integrates directly with tools like jq, aws logs, and Apache Spark.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}