How to Debug Coded Tools Using pytest.set_trace() in Neuro-SAN Studio

Use from pytest import set_trace in your test file, insert set_trace() at the desired breakpoint, and run python -m pytest <test_file> -s to drop into an interactive debugger that lets you inspect the internal state of any coded tool.

Neuro-SAN Studio, maintained by cognizant-ai-lab, provides a framework for building agent networks using coded tools—Python classes like Accountant, CalculatorTool, and TimeTool that expose custom logic to agents. When these tools misbehave or return unexpected results, you need a way to inspect their execution flow without modifying the core framework. The pytest.set_trace() helper offers a lightweight, zero-dependency method to pause test execution and examine variables directly inside the tool's runtime context.

Why Use pytest.set_trace() for Coded Tools?

While Python’s built-in pdb.set_trace() works, pytest.set_trace() offers tighter integration with the test runner used throughout Neuro-SAN Studio. Because the debugger runs in the same process as the test, you can inspect arguments passed to the tool, step through the invoke method line-by-line, and modify state on-the-fly to test edge cases.

Key advantages over standard PDB:

• Automatic pytest integration – Respects pytest’s capture settings and output handling.
• Works seamlessly with -s – No manual disabling of output capture required.
• Consistent with pytest idioms – Aligns with other helpers like pytest.raises.

Step-by-Step Debugging Workflow

Locate the Target Test

Identify the unit test that exercises the coded tool you need to debug. For example, tests/coded_tools/basic/test_accountant.py validates the Accountant tool defined in coded_tools/basic/accountant.py.

Insert the Breakpoint

Import the helper at the top of your test file and place set_trace() immediately after instantiating the tool or before calling its invoke method:

# tests/coded_tools/basic/test_accountant.py

from pytest import set_trace
from coded_tools.basic.accountant import Accountant

def test_accountant_balance():
    # Arrange – instantiate the tool

    tool = Accountant()
    # Drop into debugger right after construction

    set_trace()

    # Act – invoke the tool with a sample prompt

    result = tool.invoke("What is the balance of account 12345?")

    # Assert – verify expected output

    assert "balance" in result.lower()

Run with Interactive Output

Execute pytest with the -s flag to prevent the runner from capturing stdin/stdout, ensuring the debugger interface remains interactive:

python -m pytest tests/coded_tools/basic/test_accountant.py -s

Navigate the Debugger

When execution hits set_trace(), you’ll see a (Pdb) prompt. Use standard pdb commands to inspect the coded tool:

• p tool – Display the tool instance.
• p tool.some_internal_state – Inspect specific attributes.
• n – Execute the next line (step over).
• s – Step into a function call.
• c – Continue execution until the next breakpoint or test completion.

Practical Example: Debugging the Accountant Tool

Consider a scenario where the Accountant tool returns malformed JSON. By placing set_trace() after instantiation in tests/coded_tools/basic/test_accountant.py, you can inspect the tool’s configuration before the invoke method processes the LLM call:

def test_accountant_json_output():
    tool = Accountant()
    set_trace()  # Inspect tool's internal config here

    
    result = tool.invoke("Get account 12345 details")
    assert isinstance(result, dict)

Running with -s allows you to verify that tool is properly initialized and that any internal dictionaries or API clients contain the expected credentials before the network request is made.

Advanced Debugging Techniques

Using ipdb for Enhanced UX

For a richer debugging experience with tab-completion and syntax highlighting, configure pytest to use ipdb:

pip install ipdb
export PYTHONBREAKPOINT=ipdb.set_trace

Now set_trace() will drop into ipdb instead of standard pdb, providing colored output and auto-completion for coded tool attributes.

Debugging Parameterized Tests

When a coded tool fails only on specific input combinations, insert set_trace() inside a parameterized test to inspect each iteration:

import pytest
from pytest import set_trace
from coded_tools.basic.calculator_tool import CalculatorTool

@pytest.mark.parametrize(
    "expression,expected",
    [
        ("2+2", 4),
        ("5*3", 15),
        ("invalid", None),   # <- this case fails

    ],
)
def test_calculator(expression, expected):
    tool = CalculatorTool()
    set_trace()                # Inspect each iteration

    result = tool.invoke(expression)
    assert result == expected

Run with python -m pytest <file> -s to pause on each parameter set and identify which input triggers the failure.

Key Files for Debugging Coded Tools

File	Role
coded_tools/basic/accountant.py	Example coded tool implementation showing how tools expose logic to agents.
tests/coded_tools/basic/test_accountant.py	Unit test for Accountant—the ideal location to insert set_trace() for debugging tool behavior.
coded_tools/basic/advanced_calculator/calculator_tool.py	Numeric logic demonstration tool useful for testing parameterized debugging scenarios.
tests/coded_tools/basic/advanced_calculator/test_calculator.py	Corresponding test file for calculator logic.
pytest.ini	Pytest configuration file defining markers and coverage settings for the test suite.
run.py	Entry point for launching the Neuro-SAN Studio, useful when reproducing runtime failures in the full agent network context.

Summary

• Import set_trace from pytest, not pdb, to ensure seamless integration with the Neuro-SAN Studio test runner.
• Place breakpoints after tool instantiation in unit tests (e.g., tests/coded_tools/basic/test_accountant.py) to inspect coded tools like Accountant or CalculatorTool before they process inputs.
• Always run with -s to prevent pytest from capturing stdin/stdout, ensuring the interactive debugger remains accessible.
• Consider ipdb via PYTHONBREAKPOINT for enhanced debugging features like tab-completion when working with complex tool hierarchies.

Frequently Asked Questions

What is the difference between pytest.set_trace() and pdb.set_trace()?

pytest.set_trace() is specifically designed to work with the pytest test runner used in Neuro-SAN Studio. It automatically handles pytest’s output capture mechanisms, meaning you don’t need to manually disable capture to see the debugger prompt. In contrast, pdb.set_trace() may hang or behave unexpectedly when pytest is capturing stdout/stderr, requiring additional flags to function properly.

How do I debug a coded tool that fails only in the full agent network?

When a tool works in isolation but fails during full network execution, first reproduce the failure using the studio’s entry point in run.py. Then, create a targeted unit test in the tests/coded_tools/ directory that simulates the specific input triggering the failure. Insert set_trace() in this isolated test to inspect the tool’s state without the complexity of the full agent network running.

Can I use pytest.set_trace() with VS Code or PyCharm?

Yes, though IDE debuggers typically require additional configuration. When running tests inside VS Code or PyCharm, you can either use the IDE’s built-in "Debug Test" functionality instead of set_trace(), or configure the IDE to recognize the PDB prompt when running with -s. For the smoothest experience with set_trace(), run tests from the terminal with python -m pytest -s and use the command-line PDB interface.

Why does my debugger hang when running tests without the -s flag?

Pytest captures stdout and stderr by default to provide clean test output reports. When set_trace() triggers a PDB session, it attempts to read from stdin and write to stdout. If pytest is capturing these streams, the debugger cannot interact with your terminal, causing it to appear hung or frozen. The -s (or --capture=no) flag disables this capture, allowing the debugger direct access to the terminal I/O streams.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}