# How the Munder Difflin Circuit Breaker Recovers from Tripped States

> Learn how the Munder Difflin circuit breaker recovers incrementally from tripped states. Discover its gradual restoration process for full operation in src/main/breaker.ts.

- Repository: [Chaitanya Giri/munder-difflin](https://github.com/chaitanyagiri/munder-difflin)
- Tags: internals
- Published: 2026-08-22

---

**The Munder Difflin circuit breaker recovers incrementally by decrementing the agent's severity level one step toward "healthy" on each heartbeat tick where no trip conditions are met, gradually restoring full operation through the evaluation cycle in [`src/main/breaker.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/breaker.ts).**

The Munder Difflin circuit breaker implements a graduated recovery mechanism that prevents abrupt state resets by requiring multiple healthy evaluations to return an agent to full capacity. Located in the open-source `chaitanyagiri/munder-difflin` repository, this system processes recovery candidates on every "breaker beat," stepping down through severity levels only when telemetry confirms the underlying instability has resolved.

## Incremental Level Stepping in [`src/main/breaker.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/breaker.ts)

The core recovery logic executes during the `breaker.tick()` evaluation cycle. When the system detects no tripping conditions, it enters the recovery branch at lines 62-68:

```typescript
} else {
  // recovery – move one level down, never below ‘healthy’
  target = LEVELS[Math.max(rank(s.level) - 1, 0)]; // recover one level
}

```

This **incremental decrement** ensures an agent transitions from `constrained` to `steering` on the next healthy tick, then to `healthy` on the subsequent tick. The `rank()` function maps string severity levels to numeric indices, while `Math.max(rank(s.level) - 1, 0)` guarantees the level never drops below the `healthy` baseline (index 0). This staged approach prevents thrashing by requiring sustained stability rather than allowing instant recovery after a single benign observation.

## Telemetry Signals That Enable Recovery

Before the decrement logic can execute, the breaker must clear the counters that triggered the original escalation. The system monitors specific telemetry signals to reset internal trip conditions:

### Distinct Tool Call Detection

When an agent invokes a new combination of `toolName` and `toolInput` parameters, the `recordToolUse` method resets repetition counters. According to lines 57-66 in [`breaker.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/breaker.ts), this distinct tool call updates `lastDistinctToolAt` and zeroes both `repeatCount` and `errorCount`, removing the quantitative conditions that forced the level upward.

### Progress Indicators

The `progressing` boolean flag in the heartbeat sample indicates forward movement, such as file modification time changes. Lines 33-42 of the evaluation block clear the `noProgressBeats` debounce counter when this flag is true, signaling that the agent has resumed productive work and halting further escalation.

### Compaction Completion

When compaction ends, the system shortens the exemption window (lines 87-90), re-enabling normal velocity checks. This signal indicates resource pressure has eased, permitting the breaker to evaluate standard metrics against thresholds again.

## Immediate Recovery When Disabled

If administrators disable the circuit breaker via configuration (`cfg.enabled === false`), the system bypasses gradual stepping entirely. Lines 16-24 in [`breaker.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/breaker.ts) force every agent to the `healthy` state immediately, wiping any lingering level assignments or trip reasons. This provides an emergency override for scenarios requiring instant restoration of full agent capabilities.

## Practical Recovery Implementation

The following TypeScript example demonstrates the transition from a tripped state back to healthy operation through distinct tool usage:

```typescript
import { CircuitBreaker } from './breaker';

// Initialize breaker with configuration
const breaker = new CircuitBreaker(() => ({
  enabled: true,
  hardStop: false,
  repeatedToolLimit: 8,
  errorStormLimit: 5,
  tokenVelocityPerMin: 60_000,
}));

// Simulate tripped state via repeated identical tool calls
breaker.recordToolUse('agent-42', 'search', { query: 'foo' });
breaker.recordToolUse('agent-42', 'search', { query: 'foo' });
// ... repeat until repeatCount ≥ repeatedToolLimit

// Escalation tick moves to next severity level
let decisions = breaker.tick(
  [{ agentId: 'agent-42', sample: null, progressing: false }],
  Date.now()
);
console.log(decisions[0].state.level); // "steering" or "constrained"

// Distinct tool call resets internal counters
breaker.recordToolUse('agent-42', 'write', { text: 'new content' });

// Recovery tick decrements level one step
decisions = breaker.tick(
  [{ agentId: 'agent-42', sample: null, progressing: false }],
  Date.now()
);
console.log(decisions[0].state.level); // Steps toward "healthy"

```

## Summary

- **Gradual stepping**: Recovery decrements the `LEVELS` array index by exactly one per healthy tick, transitioning `constrained` → `steering` → `healthy` without skips.
- **Counter clearing**: Distinct tool calls, progress flags, and compaction events reset `repeatCount`, `errorCount`, and `noProgressBeats` to eliminate trip conditions.
- **Safety floors**: The `Math.max(rank(s.level) - 1, 0)` calculation ensures levels never descend below `healthy` (index 0).
- **Emergency override**: Setting `enabled: false` forces immediate return to `healthy` state according to lines 16-24 in [`breaker.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/breaker.ts).
- **Tick-driven evaluation**: All recovery state transitions occur exclusively during the `breaker.tick()` execution in [`src/main/breaker.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/breaker.ts).

## Frequently Asked Questions

### How many ticks does full recovery require?

Full recovery requires one healthy tick per severity level. An agent at `constrained` requires two consecutive healthy evaluations to reach `healthy`, while `steering` requires one. The total wall-clock time depends on the heartbeat interval configured in [`src/main/index.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/index.ts).

### What triggers immediate recovery to the healthy state?

Configuration disables trigger instant recovery. When `cfg.enabled` evaluates to `false`, lines 16-24 of [`src/main/breaker.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/breaker.ts) immediately reset all agents to `healthy`, bypassing the incremental stepping mechanism entirely and clearing all trip reasons.

### Can the recovery rate be accelerated or configured?

The repository does not expose a recovery acceleration parameter. The step-down rate is hardcoded to one level per tick in the `else` branch of the evaluation logic (lines 62-68). Operators seeking faster recovery must either disable the breaker entirely or ensure agents generate distinct tool calls or progress signals to clear trip counters immediately.

### Where is the circuit breaker recovery logic located?

The primary recovery implementation resides in [`src/main/breaker.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/breaker.ts), specifically lines 62-68 where the `target` level calculation decrements the rank. Supporting configuration definitions exist in [`src/main/config.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/config.ts), while the heartbeat integration and tick scheduling occur in [`src/main/index.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/index.ts).