# How the Debug Utility Facilitates Tracing and Logging of Agent Execution Flow in Lemon AI

> Trace and log Lemon AI agent execution flow with the debug utility. Get complete visibility using real-time console output and structured JSON logging without impacting performance.

- Repository: [hexdocom/lemonai](https://github.com/hexdocom/lemonai)
- Tags: deep-dive
- Published: 2026-03-03

---

**Lemon AI's debug utility provides a two-tier tracing system that combines real-time console output from the XML parser and stream handler with structured JSON logging via Pino, enabling complete visibility into agent execution flow without impacting production performance.**

The debug utility in Lemon AI (hexdocom/lemonai) facilitates tracing and logging of agent execution flow through a lightweight, opt-in instrumentation layer woven throughout the core XML streaming parser, stream utilities, and centralized Logger class. By toggling a simple `debug` flag or setting the `LOG_LEVEL` environment variable, developers can obtain step-by-step visibility of how an agent's actions are parsed, streamed, and recorded, without impacting runtime performance when disabled.

## Debug Flag in the Streaming XML Parser

The XML streaming parser located in [`src/xml/resolve.xml.optimize.js`](https://github.com/hexdocom/lemonai/blob/main/src/xml/resolve.xml.optimize.js) serves as the primary entry point for interpreting agent actions encoded in XML (e.g., `<write_code>`, `<finish>`). The debug utility instruments this parser to expose the exact recognition and buffering behavior of each action.

### Implementation Details

The parser's constructor accepts an `options` object where `options.debug` is stored in `this.debug` (line 32). Every internal helper—including `_log`, `_handleFieldStart`, `_handleActionStart`, and `_handleFieldClose`—checks `this.debug` before emitting console output.

When enabled, the `_log` method (lines 49-55) prints contextual information including the current tag, buffer offsets, and a preview of the surrounding XML. These logs reveal **when an action or field begins, when it ends, and how the buffer is cleaned**, providing a fine-grained view of the XML-driven execution flow.

### Execution Flow Visibility

By turning on the debug flag, developers can see exactly which actions are being recognized, how fields are streamed, and where the parser may be waiting for more data. This is essential for diagnosing malformed responses from LLMs or network truncation that could otherwise cause silent failures in agent behavior.

## Debug Output in the Generic Stream Helper

While the XML parser structures high-level actions, the stream utility in [`src/utils/stream.util.js`](https://github.com/hexdocom/lemonai/blob/main/src/utils/stream.util.js) handles the raw token flow from the language model.

The `handleStream` function receives a `debug` argument that defaults to `true`. When `debug` is true, each token received from an LLM is echoed to `process.stdout` (`debug && process.stdout.write(token);` – lines 14 and 44).

This makes it possible to trace unexpected token boundaries or premature termination that could affect downstream parsing, bridging the gap between raw model output and structured XML consumption.

## Centralized Logging via Logger.debug

For production-grade observability, Lemon AI implements a centralized logging system in [`src/logging/logger.js`](https://github.com/hexdocom/lemonai/blob/main/src/logging/logger.js).

The `Logger` class wraps the **pino** library and defines a `debug(message, data = {})` method (lines 48-50) that forwards to `pino.debug`. Throughout the agent codebase, execution milestones call `global.logging(this.context, 'AgenticAgent.run_loop', task);` (e.g., line 24 in `AgenticAgent.run_loop`).

When the application's `LOG_LEVEL` environment variable is set to `debug`, these calls emit structured JSON logs containing timestamps, task identifiers, and supplemental data. This provides **persistent, searchable logs** that can be aggregated across multiple agents and sessions, complementing the inline console traces used during development.

## Integration in the Agent Execution Pipeline

The complete debug utility comes together in [`src/agent/AgenticAgent.js`](https://github.com/hexdocom/lemonai/blob/main/src/agent/AgenticAgent.js), where the agent's main loop (`run_loop`) resolves tasks and calls `completeCodeAct`.

When a task is processed:
1. The LLM response is streamed through `handleStream`, optionally echoing tokens via the debug flag.
2. The stream is fed into the XML parser with the `debug` option passed from the agent's configuration.
3. Each step is recorded via `global.logging` calls that respect the `LOG_LEVEL` setting.

As a result, the **full lifecycle**—from LLM token arrival, through XML parsing, to task status updates—is observable either on the console (debug flag) or in structured logs (`Logger.debug`).

## Practical Implementation Examples

### Enabling the XML Parser's Debug Mode

```javascript
const { createStreamingParser } = require('@src/xml/resolve.xml.optimize');

// Turn on debugging for an agent run
const parser = createStreamingParser(
  chunk => console.log('Chunk →', chunk), // onChunk callback
  null,                                 // use default action list
  { debug: true }                       // ← enable verbose tracing
);
parser.feed(xmlResponse);

```

*Result:* The console will show messages like:

```

[XML Parser] 🆕 Action 开始: write_code
[XML Parser] 🆕 Field 开始: path
[XML Parser] ✅ Field 闭合: write_code.path { … }
...

```

### Using the Generic Stream Helper with Debug Output

```javascript
const { handleStream } = require('@src/utils/stream.util');

// Stream LLM tokens while echoing them for debugging
handleStream('sse', response, true); // true → debug flag

```

### Logging a Task Transition at Debug Level

```javascript
global.logging(this.context, 'AgenticAgent.handle_task_status', {
  taskId: task.id,
  newStatus: 'completed',
  details: result
});

```

If `process.env.LOG_LEVEL` is set to `debug`, pino will emit a JSON line similar to:

```json
{
  "level":"debug",
  "msg":"AgenticAgent.handle_task_status",
  "taskId":"c9f7a2…",
  "newStatus":"completed",
  "details":{…},
  "time":"2026-03-03T12:34:56.789Z"
}

```

## Summary

- The **XML streaming parser** ([`src/xml/resolve.xml.optimize.js`](https://github.com/hexdocom/lemonai/blob/main/src/xml/resolve.xml.optimize.js)) uses a `debug` flag to emit step-by-step console traces showing when actions and fields start, end, and how buffers are managed.
- The **stream utility** ([`src/utils/stream.util.js`](https://github.com/hexdocom/lemonai/blob/main/src/utils/stream.util.js)) provides optional real-time token echoing via the `handleStream` debug parameter, revealing raw LLM output before XML parsing.
- The **Logger class** ([`src/logging/logger.js`](https://github.com/hexdocom/lemonai/blob/main/src/logging/logger.js)) wraps pino to emit structured JSON logs when `LOG_LEVEL=debug`, enabling persistent, searchable observability across agent sessions.
- Together, these components create a **two-tier debugging system** in [`src/agent/AgenticAgent.js`](https://github.com/hexdocom/lemonai/blob/main/src/agent/AgenticAgent.js) that traces the full execution lifecycle from token arrival through task completion without impacting production performance when disabled.

## Frequently Asked Questions

### How do I enable debug mode for the XML parser in Lemon AI?

Pass `{ debug: true }` in the options object when calling `createStreamingParser()` from [`src/xml/resolve.xml.optimize.js`](https://github.com/hexdocom/lemonai/blob/main/src/xml/resolve.xml.optimize.js). This activates the internal `_log` method, which prints action starts, field boundaries, and buffer states to the console as the parser processes agent XML.

### What is the difference between console debug output and structured logging in Lemon AI?

Console debug output is generated by the XML parser and stream utility when their `debug` flags are enabled, providing immediate, inline visibility during development. Structured logging is handled by the `Logger` class in [`src/logging/logger.js`](https://github.com/hexdocom/lemonai/blob/main/src/logging/logger.js) using pino, which emits JSON-formatted records suitable for aggregation and long-term storage when `LOG_LEVEL` is set to `debug`.

### Does enabling debug mode impact production performance in Lemon AI?

No. The debug utility is designed as an opt-in, lightweight layer. When the `debug` flag is set to `false` or omitted, the parser and stream helper skip all console output operations. Similarly, the `Logger.debug` method only emits records when the environment's `LOG_LEVEL` is explicitly set to `debug`, ensuring zero overhead in standard production configurations.

### Where are the debug logs stored when using the Logger class?

The `Logger` class in [`src/logging/logger.js`](https://github.com/hexdocom/lemonai/blob/main/src/logging/logger.js) writes to `process.stdout` by default as part of its pino integration. In production deployments, these streams are typically captured by container orchestrators, log shippers, or redirected to files via shell redirection (e.g., `node app.js > app.log`). The logs are not written to a specific file path by the library itself, maintaining flexibility for various deployment environments.