How to Debug Issues Using the Tracing and Logging System in gpt-engineer

Enable the --verbose flag to surface DEBUG-level logs across all modules, or configure per-module loggers in gpt_engineer/core/ai.py and gpt_engineer/core/default/steps.py to trace specific execution paths.

The gpt-engineer repository provides a lightweight yet powerful debugging infrastructure built on Python’s standard logging module. Understanding how to leverage this tracing and logging system allows you to isolate failures in AI generation steps, monitor token consumption, and capture full stack traces when workflows fail.

Understanding the Logging Architecture in gpt-engineer

Per-Module Loggers vs. Global Configuration

The codebase uses Python’s standard pattern of creating module-level loggers via logging.getLogger(__name__). This design allows granular control over debug output. For example, in gpt_engineer/core/ai.py (lines 19–47), the AI wrapper initializes its own logger to capture request metadata and response errors without flooding other modules with noise.

Key Files in the Tracing System

Several core files implement specific logging behaviors:

• gpt_engineer/applications/cli/main.py (lines 447–448): Configures the root logger based on the --verbose CLI flag, setting logging.DEBUG when enabled or logging.INFO by default.

• gpt_engineer/core/ai.py: Logs API interactions and unexpected errors at the AI layer.

• gpt_engineer/core/token_usage.py (lines 28–30): Emits DEBUG messages showing token count calculations for budgeting and optimization.

• gpt_engineer/core/diff.py (line 189): Issues WARNING-level logs when heuristics detect ambiguous diff start indices.

• gpt_engineer/core/default/steps.py (line 387): Prints full Python stack traces to sys.stdout when step execution fails.

How to Enable Verbose Logging for Debugging

Using the --verbose CLI Flag

The simplest way to debug issues using the tracing and logging system is to append the --verbose (or -v) flag to your command:

gpt-engineer my-project --verbose

When present, the CLI configures logging.basicConfig to use logging.DEBUG, causing every module-level logger to surface DEBUG messages, including token usage calculations and AI request metadata.

Setting Log Levels via Environment Variables

While the CLI directly controls the root level via the flag, you can implement custom wrapper scripts to set specific log levels via environment variables:

export LOG_LEVEL=DEBUG
gpt-engineer my-project

This approach is useful when you need to override logging behavior in containerized environments or CI/CD pipelines without modifying CLI arguments.

Interpreting Log Output and Tracebacks

Understanding Log Levels and Their Meanings

The tracing system uses standard Python log levels to categorize output:

• DEBUG: Low-level operational details such as token counts in gpt_engineer/core/token_usage.py and API request payloads in gpt_engineer/core/ai.py.

• INFO: High-level workflow progress, such as "Generating prompt" or "Saving file X.py".

• WARNING: Non-critical anomalies, such as ambiguous diff indices in gpt_engineer/core/diff.py (line 189).

• ERROR/CRITICAL: Exceptions caught by the framework, including full stack traces printed via traceback.print_exc in gpt_engineer/core/default/steps.py.

Reading Stack Traces from steps.py

When a step fails, gpt_engineer/core/default/steps.py (line 387) catches the exception and prints a complete traceback to stdout:

traceback.print_exc(file=sys.stdout)

This output interleaves with standard logs, making it easy to identify the exact line where execution failed. To capture this for offline analysis, redirect the output:

gpt-engineer my-project --verbose 2>&1 | tee debug.log

Adding Custom Logging to Your Extensions

When extending gpt-engineer, follow the repository’s pattern of module-level loggers:

import logging
import sys
import traceback

log = logging.getLogger(__name__)

def my_custom_step():
    log.debug("Entering my_custom_step")
    try:
        # Your implementation

        pass
    except Exception:
        log.exception("Unhandled error in my_custom_step")
        traceback.print_exc(file=sys.stdout)

This ensures your debugging output respects the global --verbose flag and integrates seamlessly with the existing tracing and logging system.

Summary

• The tracing and logging system in gpt-engineer uses Python’s standard logging module with per-module loggers in files like gpt_engineer/core/ai.py and gpt_engineer/core/token_usage.py.

• Enable global debug output using the --verbose CLI flag, which sets logging.DEBUG in gpt_engineer/applications/cli/main.py (lines 447–448).

• Unhandled exceptions in step execution are surfaced via traceback.print_exc(file=sys.stdout) in gpt_engineer/core/default/steps.py (line 387).

• Custom extensions should use logging.getLogger(__name__) to ensure consistent log level inheritance and integration with the existing system.

Frequently Asked Questions

How do I capture logs to a file instead of stdout?

Redirect the combined stdout and stderr streams to a file using shell redirection or the tee command. For example: gpt-engineer my-project --verbose 2>&1 | tee debug.log. This captures both Python logging output and stack traces printed via traceback.print_exc in gpt_engineer/core/default/steps.py.

What is the difference between DEBUG and INFO levels in gpt-engineer?

DEBUG messages expose low-level operational details such as token count calculations in gpt_engineer/core/token_usage.py and raw API request metadata in gpt_engineer/core/ai.py. INFO messages indicate high-level workflow progress, such as file generation status and step completion, without exposing internal data structures.

Where are unhandled exceptions logged in the codebase?

Unhandled exceptions during step execution are caught and printed in gpt_engineer/core/default/steps.py at line 387 using traceback.print_exc(file=sys.stdout). This outputs the full Python stack trace directly to stdout, interleaving with standard log messages for immediate visibility.

Can I enable logging for only specific modules?

Yes. While the --verbose flag sets the global root logger to DEBUG, you can configure individual module loggers programmatically or via environment variables. For example, set logging.getLogger('gpt_engineer.core.ai').setLevel(logging.DEBUG) to debug only the AI wrapper in gpt_engineer/core/ai.py while keeping other modules at INFO level.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}