Schema Validation for ChatDev Workflow Configuration Contracts: A Complete Technical Guide

ChatDev validates workflow YAML files through a two-layer system: schema validation against typed dataclasses in entity/configs/graph.py and logical validation of graph structure in check/check_workflow.py.

The OpenBMB/ChatDev repository defines AI agent workflows as directed graphs via typed YAML configuration contracts. Understanding the schema validation for ChatDev workflow configuration contracts ensures your custom nodes and edges conform to the expected structure before runtime execution fails.

The Two-Layer Validation Architecture

ChatDev implements a robust validation pipeline that operates at distinct levels to guarantee workflow integrity.

Schema Validation Layer

The first layer ensures YAML files conform to the strongly-typed DesignConfig and GraphDefinition dataclasses. This validates field types, required fields, enums, and basic structural constraints. The implementation resides in check/check_yaml.py, specifically within the validate_design() function (lines 11-27).

Logical Workflow Validation Layer

The second layer verifies structural properties that cannot be expressed in static dataclass schemas alone. This includes checking for unique end nodes, sub-graph consistency, and proper node/edge references. This logic lives in check/check_workflow.py inside check_workflow_structure() (lines 19-29).

Both layers query a central schema registry that stores contract definitions for every node type, edge condition, edge processor, memory store, thinking module, and model provider.

The Schema Registry System

The registry acts as the single source of truth for all configuration contracts within the system.

Central Registry Components

Located in schema_registry/registry.py, the registry defines dataclasses like NodeSchemaSpec that capture the contract name and corresponding Python config class:

@dataclass
class NodeSchemaSpec:
    name: str
    config_cls: Type["BaseConfig"]
    summary: str | None = None
    metadata: Dict[str, Any] = field(default_factory=dict)

The registry maintains dictionaries for each contract type (_node_schemas, _edge_condition_schemas, etc.) and exposes helper functions like register_node_schema() that guard against duplicate registrations with mismatching config classes.

Node Registration Linkage

Concrete node implementations link to the schema registry through runtime/node/registry.py. When registering a new node type via register_node_type() (lines 46-68), the system automatically calls register_node_schema() to populate the central registry:

def register_node_type(...):
    node_registry.register(name, target=entry)
    register_node_schema(name, config_cls=config_cls, summary=summary)

This automatic linkage ensures the validator knows about custom nodes without manual registry updates.

Typed Configuration Contracts

The public YAML contract maps directly to Python dataclasses defined in entity/configs/graph.py (lines 27-140).

DesignConfig and GraphDefinition

The DesignConfig class serves as the top-level container with three primary fields:

• version: Optional version string
• vars: Global variable map
• graph: A GraphDefinition object

GraphDefinition holds the actual workflow structure:

• id, description, log_level, is_majority_voting
• nodes: List of node objects
• edges: List of EdgeConfig objects
• start / end: Optional explicit start/end node lists

BaseConfig Utilities

All config classes inherit from BaseConfig in entity/configs/base.py, which provides the collect_schema() export method and a generic validate() hook for custom validation logic.

Running Schema Validation

Validate your workflow configurations using either CLI tools or programmatic APIs.

CLI Validation with check_yaml

The check/check_yaml.py module provides a command-line interface for schema-only validation:

python -m check.check_yaml yaml_instance/ChatDev_v1.yaml

Under the hood, the validate_design() function loads the YAML and calls DesignConfig.from_dict():

def validate_design(data: Any, set_defaults: bool = True, fn_module_ref: Optional[str] = None) -> List[str]:
    try:
        if not isinstance(data, dict):
            raise ConfigError("YAML root must be a mapping", path="root")
        DesignConfig.from_dict(data)
        return []
    except ConfigError as exc:
        return [str(exc)]

Programmatic Validation

Integrate validation into your Python scripts:

import yaml
from check.check_yaml import validate_design

with open("workflow.yaml") as f:
    data = yaml.safe_load(f)

errors = validate_design(data)
if errors:
    raise ValueError(f"Schema validation failed: {errors}")

Logical Workflow Validation

After confirming schema compliance, verify graph-level constraints that affect runtime behavior.

Graph Structure Checks

The check/check_workflow.py module performs critical runtime-level validations:

• End-node requirement: Non-majority-voting graphs must have a unique natural end or explicit end list
• Sub-graph sanity: Verifies configs are either inline blocks or valid file references
• Node existence: Validates that start and end entries reference actual node IDs

The _analyze_graph() function (lines 60-90) implements these checks:

def _analyze_graph(graph: Dict[str, Any], base_path: str, errors: List[str]) -> None:
    if not (len(sinks) == 1):
        if end is None:
            errors.append(f"{base_path}: graph lacks a unique natural end; specify 'end' explicitly")

Running Workflow Checks

Execute full validation (schema + logical) via CLI:

python -m check.check_workflow yaml_instance/ChatDev_v1.yaml

Bulk Validation for CI Pipelines

The tools/validate_all_yamls.py script walks the yaml_instance/ directory, running both validators on every YAML file. This serves as the CI gate to ensure all example workflows remain valid:

python tools/validate_all_yamls.py

Registering Custom Node Types

When implementing new node classes, registration ensures schema validation recognizes your custom configuration:

from runtime.node.registry import register_node_type, NodeCapabilities

class MyCustomNodeConfig(BaseConfig):
    api_endpoint: str
    timeout_seconds: int

register_node_type(
    name="my_custom_node",
    config_cls=MyCustomNodeConfig,
    executor_cls=MyCustomExecutor,
    capabilities=NodeCapabilities(exposes_tools=True),
    summary="Custom API-calling node"
)

The register_node_type call automatically creates a NodeSchemaSpec entry, making the node available to both the validator and runtime engine.

Summary

• Schema validation enforces type safety through DesignConfig.from_dict() in check/check_yaml.py, verifying field types and required values.
• Logical validation ensures graph structural integrity via check_workflow_structure() in check/check_workflow.py, checking end-node uniqueness and sub-graph consistency.
• The central schema registry in schema_registry/registry.py maps contract names to Python classes, enabling automatic validation of custom nodes.
• Use python -m check.check_yaml for quick schema checks and python -m check.check_workflow for full validation.
• Register new node types through runtime/node/registry.py to automatically include them in the validation pipeline.

Frequently Asked Questions

What is the difference between schema validation and logical validation in ChatDev?

Schema validation checks that YAML files conform to the typed DesignConfig hierarchy—field types, required fields, and enum values—using check/check_yaml.py. Logical validation verifies runtime graph properties like unique end nodes and proper sub-graph references via check/check_workflow.py. Schema validation catches syntax errors immediately, while logical validation prevents runtime workflow execution failures.

How do I register a custom node type for schema validation?

Import register_node_type from runtime/node/registry.py and call it with your node name, config class, and executor class. This automatically adds a NodeSchemaSpec to the central registry via register_node_schema(), making your custom node available to the validator without manual registry manipulation.

What are the common schema validation errors when authoring ChatDev workflows?

Common failures include missing required fields in GraphDefinition, type mismatches in node configurations (e.g., providing a string where an integer is expected), and malformed YAML root structures. The validator returns human-readable paths like graph.end to pinpoint exact error locations.

How can I validate all workflow files in a directory at once?

Run python tools/validate_all_yamls.py from the repository root. This script recursively walks the yaml_instance/ directory and executes both validate_design() and check_workflow_structure() on every YAML file, making it ideal for CI pipelines and pre-commit hooks.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}