How to Implement the Circuit Breaker Pattern for Fault Tolerance in Easegress

Easegress implements the circuit breaker pattern through a Resilience Policy that wraps proxy handlers with a state machine—monitoring failure rates and response times to automatically reject requests when thresholds are exceeded, preventing cascading failures downstream.

The circuit breaker pattern is essential for building resilient microservices architectures by preventing cascading failures when downstream services become unhealthy. In the megaease/easegress open-source API gateway, this fault tolerance mechanism is built directly into the proxy layer through configurable Resilience Policies. By leveraging the native circuit breaker implementation, you can protect your HTTP and gRPC traffic without writing custom middleware or modifying application code.

How Easegress Implements the Circuit Breaker Mechanism

The core logic lives in pkg/util/circuitbreaker/circuitbreaker.go, which maintains a state machine with four possible states: Closed (normal operation), Open (rejecting requests), Half-Open (testing recovery), and ForceOpen/Disabled (manual override).

The integration layer in pkg/resilience/circuitbreaker.go bridges the declarative policy configuration with the core library by creating a circuitBreakerWrapper. This wrapper intercepts every request through the following lifecycle:

1. AcquirePermission: Validates the current state before execution. If the breaker is Open, it immediately returns ErrShortCircuited.
2. Execute Handler: Runs the actual request while measuring latency.
3. RecordResult: Feeds success, failure, or "slow call" outcomes into a sliding window (count-based or time-based).
4. Transition State: Evaluates thresholds (failure-rate, slow-call-rate, minimum calls) to transition between Closed → Open → Half-Open → Closed.

In pkg/filters/proxies/httpproxy/pool.go, the ServerPool.InjectResiliencePolicy method looks up the policy by name and injects the wrapper into the request pipeline:

if name := sp.spec.CircuitBreakerPolicy; name != "" {
    p := policies[name]
    sp.circuitBreakerWrapper = policy.CreateWrapper()
}

When processing requests, ServerPool.handle wraps the handler transparently. If ErrShortCircuited is returned, the pool automatically generates a 503 Service Unavailable response.

Configuring a Circuit Breaker Policy

Define a CircuitBreaker resource (YAML) to specify thresholds and window behavior. The policy uses a sliding window to track recent call outcomes and determine when to open the circuit.

apiVersion: easegress.io/v2
kind: CircuitBreaker
metadata:
  name: example-cb
spec:
  failureRateThreshold: 50          # Percentage of failures to trigger Open state

  slowCallRateThreshold: 100        # Percentage of slow calls (optional)

  slidingWindowType: "COUNT_BASED"   # Options: COUNT_BASED or TIME_BASED

  slidingWindowSize: 100            # Number of calls (count) or seconds (time) to track

  permittedNumberOfCallsInHalfOpen: 10
  minimumNumberOfCalls: 100         # Minimum calls before calculating error rates

  slowCallDurationThreshold: "1m"   # Threshold for considering a call "slow"

  waitDurationInOpen: "1m"          # Duration to wait before transitioning to Half-Open

Key parameters explained:

• failureRateThreshold: When the error rate exceeds this percentage within the sliding window, the breaker opens.
• slidingWindowType: COUNT_BASED tracks the last N calls; TIME_BASED tracks all calls within the last N seconds.
• minimumNumberOfCalls: Prevents the breaker from opening due to a single failure during low traffic.

Attaching the Policy to Proxies

Reference the policy name in your proxy configuration using the circuitBreakerPolicy field. This works identically for both HTTP and gRPC proxies.

HTTP Proxy example:

apiVersion: easegress.io/v2
kind: HTTPProxy
metadata:
  name: my-service-proxy
spec:
  # ... other configuration fields ...

  circuitBreakerPolicy: example-cb

gRPC Proxy: The same injection logic applies to gRPC services via pkg/filters/proxies/grpcproxy/pool.go. Specify circuitBreakerPolicy in your GRPCProxy spec to enable fault tolerance for gRPC traffic.

Deploy the configuration using the Easegress CLI:

easegressctl apply -f circuit-breaker.yaml

Once deployed, the server pool automatically wraps all handlers; no application code changes are required.

Advanced Usage and Observability

For custom components or programmatic control, you can instantiate the circuit breaker directly using the internal library.

Manual Circuit Breaker Creation

import (
    libcb "github.com/megaease/easegress/v2/pkg/util/circuitbreaker"
    "time"
)

// Configure policy with 50% failure threshold and count-based window
policy := &libcb.Policy{
    FailureRateThreshold:             50,
    SlowCallRateThreshold:            100,
    SlidingWindowType:                libcb.CountBased,
    SlidingWindowSize:                100,
    PermittedNumberOfCallsInHalfOpen: 10,
    MinimumNumberOfCalls:             100,
    SlowCallDurationThreshold:        time.Second,
    WaitDurationInOpen:               time.Minute,
}

cb := libcb.New(policy)

// Wrap business logic
wrapped := func(ctx context.Context) error {
    permitted, stateID := cb.AcquirePermission()
    if !permitted {
        return libcb.ErrRejected // Request short-circuited
    }
    
    start := time.Now()
    err := myBusinessLogic(ctx)
    cb.RecordResult(stateID, err != nil, time.Since(start))
    return err
}

Monitoring State Transitions

Implement an EventListenerFunc to capture state changes for metrics or alerting:

breaker.SetStateListener(func(e *libcb.Event) {
    log.Printf("Circuit breaker transitioned from %s to %s, reason: %s",
        e.OldState, e.NewState, e.Reason)
})

The Event structure provides timestamps, previous and new state strings, and the transition reason, enabling integration with Prometheus, Grafana, or custom logging pipelines.

Summary

• Core Implementation: The circuit breaker logic resides in pkg/util/circuitbreaker/circuitbreaker.go, while policy parsing and wrapper creation occur in pkg/resilience/circuitbreaker.go.
• Proxy Integration: Attach policies via the circuitBreakerPolicy field in HTTPProxy or GRPCProxy specs; the server pool in pkg/filters/proxies/httpproxy/pool.go (or grpcproxy/pool.go) automatically injects the wrapper.
• State Management: The breaker cycles through Closed, Open, and Half-Open states based on configurable failure-rate and slow-call-rate thresholds tracked in sliding windows.
• Automatic Failures: When the circuit is open, Easegress returns HTTP 503 Service Unavailable without forwarding requests to unhealthy backends.
• Observability: Hook into state transition events using SetStateListener for real-time monitoring of fault tolerance behavior.

Frequently Asked Questions

What happens when the circuit breaker opens?

When the failure rate or slow-call rate exceeds the configured thresholds, the breaker transitions to the Open state. In this state, AcquirePermission returns false, causing the wrapper to return ErrShortCircuited immediately. The HTTP server pool catches this error and responds with 503 Service Unavailable, protecting the downstream service from additional load while it recovers.

How do I choose between COUNT_BASED and TIME_BASED sliding windows?

Use COUNT_BASED when you want the breaker to react to error rates within a fixed number of recent requests (e.g., 100 calls), regardless of time elapsed. Use TIME_BASED when traffic patterns vary significantly and you want to evaluate error rates within a specific time window (e.g., the last 60 seconds), ensuring the breaker considers recent temporal behavior rather than absolute call volume.

Can I use circuit breakers with gRPC services?

Yes. The circuit breaker implementation is protocol-agnostic at the wrapper level. For gRPC proxies, the GRPCProxy resource supports the same circuitBreakerPolicy field as HTTPProxy. The injection logic in pkg/filters/proxies/grpcproxy/pool.go mirrors the HTTP implementation, wrapping gRPC handlers to provide identical fault tolerance semantics.

How can I force the circuit breaker to remain open or closed?

The CircuitBreakerPolicy supports ForceOpen and Disabled states via configuration, though typically you control this through the standard threshold parameters. For manual control in custom code, you can interact with the underlying libcb.CircuitBreaker instance directly and manipulate state transition logic, or simply set extreme threshold values (e.g., failureRateThreshold: 0 to never open, or failureRateThreshold: 1 with minimumNumberOfCalls: 1 to open immediately on any error).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}