How the Debug Utility Facilitates Tracing and Logging of Agent Execution Flow in Lemon AI

Lemon AI's debug utility provides a two-tier tracing system that combines real-time console output from the XML parser and stream handler with structured JSON logging via Pino, enabling complete visibility into agent execution flow without impacting production performance.

The debug utility in Lemon AI (hexdocom/lemonai) facilitates tracing and logging of agent execution flow through a lightweight, opt-in instrumentation layer woven throughout the core XML streaming parser, stream utilities, and centralized Logger class. By toggling a simple debug flag or setting the LOG_LEVEL environment variable, developers can obtain step-by-step visibility of how an agent's actions are parsed, streamed, and recorded, without impacting runtime performance when disabled.

Debug Flag in the Streaming XML Parser

The XML streaming parser located in src/xml/resolve.xml.optimize.js serves as the primary entry point for interpreting agent actions encoded in XML (e.g., <write_code>, <finish>). The debug utility instruments this parser to expose the exact recognition and buffering behavior of each action.

Implementation Details

The parser's constructor accepts an options object where options.debug is stored in this.debug (line 32). Every internal helper—including _log, _handleFieldStart, _handleActionStart, and _handleFieldClose—checks this.debug before emitting console output.

When enabled, the _log method (lines 49-55) prints contextual information including the current tag, buffer offsets, and a preview of the surrounding XML. These logs reveal when an action or field begins, when it ends, and how the buffer is cleaned, providing a fine-grained view of the XML-driven execution flow.

Execution Flow Visibility

By turning on the debug flag, developers can see exactly which actions are being recognized, how fields are streamed, and where the parser may be waiting for more data. This is essential for diagnosing malformed responses from LLMs or network truncation that could otherwise cause silent failures in agent behavior.

Debug Output in the Generic Stream Helper

While the XML parser structures high-level actions, the stream utility in src/utils/stream.util.js handles the raw token flow from the language model.

The handleStream function receives a debug argument that defaults to true. When debug is true, each token received from an LLM is echoed to process.stdout (debug && process.stdout.write(token); – lines 14 and 44).

This makes it possible to trace unexpected token boundaries or premature termination that could affect downstream parsing, bridging the gap between raw model output and structured XML consumption.

Centralized Logging via Logger.debug

For production-grade observability, Lemon AI implements a centralized logging system in src/logging/logger.js.

The Logger class wraps the pino library and defines a debug(message, data = {}) method (lines 48-50) that forwards to pino.debug. Throughout the agent codebase, execution milestones call global.logging(this.context, 'AgenticAgent.run_loop', task); (e.g., line 24 in AgenticAgent.run_loop).

When the application's LOG_LEVEL environment variable is set to debug, these calls emit structured JSON logs containing timestamps, task identifiers, and supplemental data. This provides persistent, searchable logs that can be aggregated across multiple agents and sessions, complementing the inline console traces used during development.

Integration in the Agent Execution Pipeline

The complete debug utility comes together in src/agent/AgenticAgent.js, where the agent's main loop (run_loop) resolves tasks and calls completeCodeAct.

When a task is processed:

1. The LLM response is streamed through handleStream, optionally echoing tokens via the debug flag.
2. The stream is fed into the XML parser with the debug option passed from the agent's configuration.
3. Each step is recorded via global.logging calls that respect the LOG_LEVEL setting.

As a result, the full lifecycle—from LLM token arrival, through XML parsing, to task status updates—is observable either on the console (debug flag) or in structured logs (Logger.debug).

Practical Implementation Examples

Enabling the XML Parser's Debug Mode

const { createStreamingParser } = require('@src/xml/resolve.xml.optimize');

// Turn on debugging for an agent run
const parser = createStreamingParser(
  chunk => console.log('Chunk →', chunk), // onChunk callback
  null,                                 // use default action list
  { debug: true }                       // ← enable verbose tracing
);
parser.feed(xmlResponse);

Result: The console will show messages like:

[XML Parser] 🆕 Action 开始: write_code
[XML Parser] 🆕 Field 开始: path
[XML Parser] ✅ Field 闭合: write_code.path { … }
...

Using the Generic Stream Helper with Debug Output

const { handleStream } = require('@src/utils/stream.util');

// Stream LLM tokens while echoing them for debugging
handleStream('sse', response, true); // true → debug flag

Logging a Task Transition at Debug Level

global.logging(this.context, 'AgenticAgent.handle_task_status', {
  taskId: task.id,
  newStatus: 'completed',
  details: result
});

If process.env.LOG_LEVEL is set to debug, pino will emit a JSON line similar to:

{
  "level":"debug",
  "msg":"AgenticAgent.handle_task_status",
  "taskId":"c9f7a2…",
  "newStatus":"completed",
  "details":{…},
  "time":"2026-03-03T12:34:56.789Z"
}

Summary

• The XML streaming parser (src/xml/resolve.xml.optimize.js) uses a debug flag to emit step-by-step console traces showing when actions and fields start, end, and how buffers are managed.
• The stream utility (src/utils/stream.util.js) provides optional real-time token echoing via the handleStream debug parameter, revealing raw LLM output before XML parsing.
• The Logger class (src/logging/logger.js) wraps pino to emit structured JSON logs when LOG_LEVEL=debug, enabling persistent, searchable observability across agent sessions.
• Together, these components create a two-tier debugging system in src/agent/AgenticAgent.js that traces the full execution lifecycle from token arrival through task completion without impacting production performance when disabled.

Frequently Asked Questions

How do I enable debug mode for the XML parser in Lemon AI?

Pass { debug: true } in the options object when calling createStreamingParser() from src/xml/resolve.xml.optimize.js. This activates the internal _log method, which prints action starts, field boundaries, and buffer states to the console as the parser processes agent XML.

What is the difference between console debug output and structured logging in Lemon AI?

Console debug output is generated by the XML parser and stream utility when their debug flags are enabled, providing immediate, inline visibility during development. Structured logging is handled by the Logger class in src/logging/logger.js using pino, which emits JSON-formatted records suitable for aggregation and long-term storage when LOG_LEVEL is set to debug.

Does enabling debug mode impact production performance in Lemon AI?

No. The debug utility is designed as an opt-in, lightweight layer. When the debug flag is set to false or omitted, the parser and stream helper skip all console output operations. Similarly, the Logger.debug method only emits records when the environment's LOG_LEVEL is explicitly set to debug, ensuring zero overhead in standard production configurations.

Where are the debug logs stored when using the Logger class?

The Logger class in src/logging/logger.js writes to process.stdout by default as part of its pino integration. In production deployments, these streams are typically captured by container orchestrators, log shippers, or redirected to files via shell redirection (e.g., node app.js > app.log). The logs are not written to a specific file path by the library itself, maintaining flexibility for various deployment environments.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}