How Does Error Decorrelation Work in the Critic Model?

Error decorrelation in the ADHD critic model works by executing the scoring and clustering stages on a different language model family than the generator, preventing systematic biases or hallucination patterns from the generation phase from contaminating the evaluation results.

The ADHD repository implements a dual-stage pipeline that separates idea creation from idea evaluation. Understanding how error decorrelation works in the critic model allows you to configure the system for greater robustness by ensuring that model-specific errors do not propagate through both the divergent generation and convergent criticism phases.

The Dual-Stage Architecture (Generator and Critic)

The ADHD engine operates through two distinct passes: a generator that produces raw ideas and a critic that evaluates them. The generator runs using the model specified by --model (or the default SDK model), outputting a list of potential solutions for a given problem.

The critic then receives these ideas and executes two evaluation phases: scoreIdeas and clusterIdeas. Because the critic can be invoked with a different model via the --critic-model flag, any systematic bias or hallucination pattern present in the generator’s architecture does not automatically propagate to the scoring stage. This separation is the core mechanism of error decorrelation, as implemented in src/engine.ts.

Configuring Error Decorrelation via Model Separation

Error decorrelation is achievable because the system allows the critic to run on a distinct model family. A comment in src/engine.ts explicitly notes the intent to "Use a different family to decorrelate critic errors," referencing line 64 of the source.

CLI Configuration with --critic-model

The command-line interface in src/cli.ts exposes the --critic-model option, which is documented as a mechanism to "decorrelate critic errors." When provided, this flag overrides the default behavior where the critic inherits the generator's model.

# Generate with GPT-4o but evaluate with Claude to decorrelate errors

adhd run --problem "Design a reminder system" \
          --model gpt-4o \
          --critic-model claude-3-opus

Engine Implementation in src/engine.ts

Inside the engine core, the critic model selection logic appears at line 329, implementing a null-coalescing fallback pattern:

// Select critic model, defaulting to generator model if unspecified
const critic = criticModel ?? model;

This expression ensures that when criticModel is undefined, the system maintains backward compatibility by using the generator model. However, when a user explicitly supplies a different model family via the CLI, the critic variable holds a distinct model instance, creating the separation necessary for error decorrelation.

Technical Implementation Details

The critic execution phase respects the selected model configuration throughout both asynchronous evaluation stages.

The Critic Selection Logic

The engine imports the criticModel parameter from the type definitions in src/types.ts. The configuration allows for complete model overrides, meaning you can pair any generator model with any critic model supported by the SDK. This flexibility ensures that architectural differences between model families—such as variations in training data or fine-tuning objectives—create independent error distributions across the two pipeline stages.

Scoring and Clustering Execution

Once the critic model is determined, the engine executes both evaluation phases concurrently using Promise.all. Both scoreIdeas and clusterIdeas receive the selected critic model as their final argument:

// Run both critic phases on the potentially different model
await Promise.all([
  scoreIdeas(problem, allIdeas, critic),
  clusterIdeas(problem, allIdeas, critic),
]);

By passing the potentially distinct critic instance to both functions, the system ensures that neither the scoring nor the clustering operations are influenced by the generator's specific failure modes. This implementation guarantees that systematic errors in idea generation remain uncorrelated with errors in idea evaluation.

Summary

• Error decorrelation prevents model-specific hallucinations and biases from propagating between the generator and critic stages in the ADHD pipeline.
• The --critic-model CLI flag in src/cli.ts enables explicit selection of a different model family for the evaluation pass.
• src/engine.ts implements the selection logic const critic = criticModel ?? model; at line 329, allowing seamless fallback when no critic model is specified.
• The critic executes both scoreIdeas and clusterIdeas using the selected model, ensuring independent error distributions across generation and evaluation.
• Configuration requires no code modifications—users simply specify different models via command-line arguments to achieve robustness through architectural separation.

Frequently Asked Questions

What is error decorrelation in machine learning systems?

Error decorrelation is an architectural pattern that involves using independent components—or in this case, different language model families—to perform sequential tasks. In the ADHD repository, this means the generator might produce biased or hallucinated ideas due to its specific training, while the critic, running on a different model with different inductive biases, is less likely to reinforce those same errors when scoring and clustering the outputs.

Why should I use a different model family for the critic pass?

Using a different model family prevents error propagation chains. If the generator exhibits systematic blind spots—for example, consistently generating insecure code patterns due to its training distribution—a critic from a different family (such as switching from an OpenAI model to an Anthropic model) will likely evaluate the ideas against different safety heuristics. This diversity catches errors that a same-model critic might overlook, improving the reliability of the final selected output.

How do I configure error decorrelation in my ADHD workflow?

Configure error decorrelation by specifying the --critic-model flag when executing the adhd run command. For example:

adhd run --problem "Optimize SQL queries" \
          --model gpt-4o \
          --critic-model claude-3-sonnet

If you omit the --critic-model argument, the expression criticModel ?? model evaluates to the generator model, and both stages execute on identical instances, eliminating the decorrelation benefit.

What happens if I don't specify a critic model?

When no critic model is provided, the system defaults to using the generator model for both passes. As defined in src/engine.ts, the critic variable assignment falls back to the generator model, meaning any systematic errors, biases, or hallucination patterns inherent to that specific model will affect both idea creation and idea evaluation. This reduces the pipeline's ability to self-correct through independent verification.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}