How to Write Tests for Neuro SAN Agent Networks: A Complete Guide

You write tests for Neuro SAN agent networks by using the DynamicHoconUnitTests harness to drive a live server instance against HOCON fixtures, asserting expected responses, tool calls, and memory updates directly within the configuration files.

Neuro SAN agent networks are defined declaratively using HOCON configuration files, and testing them requires validating dynamic conversational behavior rather than static code paths. The cognizant-ai-lab/neuro-san-studio repository provides a robust testing framework that lets you write data-driven integration tests using parameterized pytest functions and encode assertions directly inside your HOCON fixtures.

Understanding the Neuro SAN Test Architecture

Neuro SAN tests operate differently from traditional unit tests because agent networks are orchestrated by a server process. The testing strategy relies on three core components:

• HOCON Fixtures: Configuration files that define the network topology, tools, and test expectations.
• DynamicHoconUnitTests: A reusable harness class that starts a Neuro SAN server, loads a HOCON file, sends a query, and captures the full response including tool calls and memory updates.
• Parameterized Test Methods: pytest functions that iterate over lists of HOCON fixtures, enabling you to add new test scenarios by simply adding files rather than writing new Python code.

The canonical integration test driver lives at tests/integration/test_integration_test_hocons.py, which demonstrates the standard pattern for writing tests for Neuro SAN agent networks.

Writing Integration Tests for Agent Networks

Step 1: Choose or Create a HOCON Fixture

A HOCON file describes the entire network—agents, tools, metadata, and instructions. The repository includes sample networks like registries/basic/coffee_finder_advanced.hocon, which serves as a template for new fixtures.

If you need a custom scenario, copy an existing fixture and modify the tools, prompts, or metadata. Store your test-specific fixtures under tests/fixtures/ (or any folder referenced by path_to_basis in your test class) so the harness can locate them relative to the test file.

Step 2: Set Up the Test Class with DynamicHoconUnitTests

Create a new test file under tests/integration/ and inherit from TestCase and FailFastParamMixin (the latter stops the suite on the first failure, matching the existing integration test behavior).

# tests/integration/test_my_network.py

from unittest import TestCase
import pytest
from neuro_san.test.unittest.dynamic_hocon_unit_tests import DynamicHoconUnitTests
from parameterized import parameterized
from tests.utils.fail_fast_param_mixin import FailFastParamMixin


class TestMyNetwork(TestCase, FailFastParamMixin):
    """
    Data-driven tests for a custom agent network.
    """
    DYNAMIC = DynamicHoconUnitTests(
        __file__,                     # location of this test file

        path_to_basis="../fixtures"   # folder containing HOCON fixtures

    )

    @parameterized.expand(
        DynamicHoconUnitTests.from_hocon_list(
            [
                "my_custom_network.hocon",   # new fixture

            ]
        ),
        skip_on_empty=True,
    )
    @pytest.mark.integration
    def test_my_network(self, test_name: str, test_hocon: str):
        """
        Executes the network defined in *test_hocon* and asserts that
        the generated response meets expectations.
        """
        self.DYNAMIC.one_test_hocon(self, test_name, test_hocon)

The DynamicHoconUnitTests class handles server startup, query injection, and response capture. The one_test_hocon method runs the single scenario and raises an AssertionError if the response diverges from expectations.

Step 3: Encode Test Expectations in HOCON

The test harness looks for special keys inside the HOCON file to determine what to assert:

• expected_response – exact text the network should return.
• expected_tools – list of tool calls (name + parameters) that must appear.
• expected_memory – snippets that should be stored via tools like UserPreferences.

Add these keys to your fixture:

expected_response = "Here is the coffee you ordered, Olivier: Black coffee (order 101)."
expected_tools = [
  { name = "CoffeeFinder", params = {} },
  { name = "OrderAPI", params = { shop_name = "Bob's Coffee Shop", customer_name = "Olivier", order_details = "Black coffee" } }
]
expected_memory = [
  { topic = "Olivier", new_fact = "prefers black coffee" }
]

The test runner automatically compares the live output against these values, validating both the conversational response and the internal tool orchestration.

Step 4: Run the Test Suite

Activate your virtual environment and install the development dependencies:

source venv/bin/activate
pip install -r requirements.txt
pip install -r requirements-build.txt

Execute your new test:

pytest -k test_my_network

All integration tests are marked with pytest.mark.integration, so you can run the full set with:

pytest -m integration

Unit Testing Individual Coded Tools

When you need to test a single Python-based tool (e.g., OrderAPI) without spinning up the full Neuro SAN server, use standard unittest or pytest patterns.

The repository provides an example at tests/coded_tools/basic/coffee_finder_advanced/test_order_api.py:

from coded_tools.basic.coffee_finder_advanced.order_api import OrderAPI


def test_invoke():
    order_api = OrderAPI()
    order = {
        "customer_name": "Olivier",
        "shop_name": OrderAPI.SHOP_1,
        "order_details": "Black coffee"
    }
    response = order_api.invoke(args=order, sly_data={})
    assert response == f"Order 101 placed successfully for Olivier at {OrderAPI.SHOP_1}. Details: Black coffee"

Mock external dependencies (HTTP calls, databases) using pytest-mock or unittest.mock, following the patterns demonstrated in the now_agents unit tests.

Troubleshooting Common Testing Issues

Symptom	Likely Cause	Fix
Test hangs indefinitely	DynamicHoconUnitTests cannot start the Neuro SAN server (port already in use)	Stop stray server processes (`ps aux
AssertionError on response mismatch	expected_response key missing or contains typos/whitespace differences	Double-check the exact string in your HOCON fixture; whitespace matters.
Tool call not recorded	Tool class not importable (class = "my_pkg.MyTool")	Ensure the coded tool module is in PYTHONPATH and listed in requirements.txt.
Memory assertion fails	UserPreferences tool not invoked by the agent	Verify the agent's instructions explicitly reference the UserPreferences tool name, as shown in registries/basic/coffee_finder_advanced.hocon.

Summary

Writing tests for Neuro SAN agent networks relies on the DynamicHoconUnitTests harness to bridge declarative HOCON configurations with executable assertions. Key takeaways include:

• Use HOCON fixtures to define both the network topology and the expected behavior (expected_response, expected_tools, expected_memory).
• Leverage DynamicHoconUnitTests in tests/integration/ to automatically start the Neuro SAN server, execute queries, and validate responses.
• Write isolated unit tests for individual coded tools using standard pytest patterns, mocking external dependencies as needed.
• Reference canonical examples like tests/integration/test_integration_test_hocons.py and registries/basic/coffee_finder_advanced.hocon when creating new test scenarios.

Frequently Asked Questions

How do I add a new test scenario without writing Python code?

You can add a new test scenario by creating a HOCON fixture file and adding it to the list passed to DynamicHoconUnitTests.from_hocon_list(). The parameterized.expand decorator will automatically pick up the new file and generate a test case for it. No additional Python code is required unless you need custom assertion logic.

What is the difference between integration tests and unit tests in Neuro SAN?

Integration tests use DynamicHoconUnitTests to spin up a full Neuro SAN server instance and validate end-to-end agent behavior through HOCON fixtures. Unit tests target individual coded tools (Python classes implementing specific logic) in isolation, using standard pytest or unittest patterns without starting the server.

How do I mock external API calls in coded tool unit tests?

Use unittest.mock or pytest-mock to patch the external client within your tool's invoke method. The repository's now_agents unit tests demonstrate this pattern, showing how to mock HTTP requests or database connections so your tests run quickly and deterministically without network dependencies.

Why does my integration test fail with a port conflict error?

The DynamicHoconUnitTests harness attempts to start a Neuro SAN server on a specific port. If another process is already using that port, the server fails to start and the test hangs or errors. Stop any running server processes with ps aux | grep neuro_san and kill them, or modify the port configuration in run.py before executing tests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}