# How Easegress Health Checks Work: A Deep Dive into Backend Service Monitoring > Discover how Easegress health checks monitor backend services using declarative specs and integrated load balancing for optimal reliability. Learn about its robust proxy infrastructure. - Repository: [easegress-io/easegress](https://github.com/easegress-io/easegress) - Tags: deep-dive - Published: 2026-03-01 --- **Easegress implements health checking as a built-in component of its proxy infrastructure, using declarative specifications, concrete HTTP/WebSocket health-checker implementations, and load-balancer integration to continuously monitor and filter healthy backend services.** The `easegress-io/easegress` repository treats health checking as a first-class feature within its proxy and load-balancing layer. Understanding how Easegress health checks function requires examining three tightly coupled parts: the **health-check specification** that defines probe parameters, the **health-checker implementations** that execute HTTP or WebSocket probes, and the **load-balancer integration** that periodically evaluates backend health and rebuilds the active server pool. ## Easegress Health Check Architecture Overview The architecture separates configuration from execution. A declarative spec describes how to probe a service, while concrete checker objects handle the actual network operations. The `GeneralLoadBalancer` orchestrates the process by running these checks on a timer and maintaining the pool of healthy backends that the request router uses. This design isolates probing logic from routing, enables fine-grained control over intervals, timeouts, and failure thresholds, and guarantees that traffic is only sent to services that have recently responded successfully. ## Health Check Specifications in Easegress All health-check settings are defined by `HealthCheckSpec` in [`pkg/filters/proxies/healthcheck.go`](https://github.com/easegress-io/easegress/blob/main/pkg/filters/proxies/healthcheck.go). Protocol-specific specs extend this base structure to add probe-specific parameters. ### Common Health Check Configuration The base specification contains timing and threshold parameters used by all checker types: ```go type HealthCheckSpec struct { Interval string `json:"interval,omitempty" jsonschema:"format=duration"` Timeout string `json:"timeout,omitempty" jsonschema:"format=duration"` Fails int `json:"fails,omitempty" jsonschema:"minimum=1"` Passes int `json:"passes,omitempty" jsonschema:"minimum=1"` Path string `json:"path,omitempty"` // deprecated, HTTP uses URI } ``` - **Interval**: How often to probe each backend. - **Timeout**: Maximum time to wait for a response. - **Fails**: Consecutive failures required to mark a server unhealthy. - **Passes**: Consecutive successes required to mark a server healthy again. ### HTTP-Specific Health Check Settings HTTP health checks extend the base spec in [`pkg/filters/proxies/httpproxy/healthcheck.go`](https://github.com/easegress-io/easegress/blob/main/pkg/filters/proxies/httpproxy/healthcheck.go) with request construction and response validation parameters: ```go type HTTPHealthCheckSpec struct { Port int `json:"port,omitempty"` URI string `json:"uri,omitempty"` Method string `json:"method,omitempty"` Headers map[string]string `json:"headers,omitempty"` Body string `json:"body,omitempty"` Match *HealthCheckMatch `json:"match,omitempty"` } ``` The **Match** field allows complex validation rules including status code ranges, header presence, and body content regex matching. ### WebSocket Health Check Configuration WebSocket health checks in the same file can combine an HTTP probe with a WebSocket handshake: ```go type WSProxyHealthCheckSpec struct { proxies.HealthCheckSpec `json:",inline"` HTTP *HTTPHealthCheckSpec `json:"http,omitempty"` WS *WSHealthCheckSpec `json:"ws,omitempty"` } ``` This flexibility allows administrators to validate both the HTTP layer and the WebSocket upgrade capability. ## Health Checker Implementations Concrete implementations of the `HealthChecker` interface execute the actual network probes. The interface is defined in [`pkg/filters/proxies/healthcheck.go`](https://github.com/easegress-io/easegress/blob/main/pkg/filters/proxies/healthcheck.go): ```go type HealthChecker interface { BaseSpec() HealthCheckSpec Check(svr *Server) bool Close() } ``` ### HTTP Health Checker The `httpHealthChecker` struct in [`pkg/filters/proxies/httpproxy/healthcheck.go`](https://github.com/easegress-io/easegress/blob/main/pkg/filters/proxies/httpproxy/healthcheck.go) sends HTTP requests built from the specification. It uses a dedicated `http.Client` with configurable timeout from `spec.GetTimeout()`. After obtaining a response, the `Match` object validates status codes, headers, and body content against the configured rules. The `Check` method returns `true` only if all match conditions succeed. Construction happens via `NewHTTPHealthChecker`: ```go func NewHTTPHealthChecker(tlsConfig *tls.Config, spec *ProxyHealthCheckSpec) proxies.HealthChecker { … } ``` ### WebSocket Health Checker The `wsHealthChecker` optionally runs an HTTP health-check first, then upgrades to a WebSocket connection using `gorilla/websocket.Dialer` with the same timeout configuration. After a successful dial, it validates the handshake response (HTTP 101) using the same `Match` validation logic. This ensures the WebSocket endpoint is both reachable and correctly performing the protocol upgrade. Construction uses `NewWebSocketHealthChecker`: ```go func NewWebSocketHealthChecker(spec *WSProxyHealthCheckSpec) proxies.HealthChecker { … } ``` ## Load Balancer Integration and Health State Management The `GeneralLoadBalancer` in [`pkg/filters/proxies/loadbalance.go`](https://github.com/easegress-io/easegress/blob/main/pkg/filters/proxies/loadbalance.go) orchestrates the health checking lifecycle. It owns a `HealthChecker` instance (`glb.hc`) and its specification (`glb.hcSpec`). ### Periodic Health Check Execution During initialization, the load balancer: 1. Retrieves the base spec from the checker to ensure sensible defaults for `Fails` and `Passes` thresholds. 2. Starts a ticker using `spec.GetInterval()` and launches a goroutine that periodically calls `glb.checkServers()`. ```go ticker := time.NewTicker(spec.GetInterval()) glb.done = make(chan struct{}) glb.checkServers() go func() { for { select { case <-glb.done: ticker.Stop() return case <-ticker.C: glb.checkServers() } } }() ``` ### Server Health State Transitions The `checkServers` method iterates over every backend `Server`: - It calls `glb.hc.Check(svr)`, which returns `true` for healthy probes and `false` for unhealthy ones. - It updates `svr.HealthCounter`, incrementing for successes and decrementing for failures. - When the counter crosses the thresholds defined by `hcSpec.Passes` or `hcSpec.Fails`, the server’s `Unhealth` flag is toggled, and the global `glb.healthyServers` pool is rebuilt. Only servers where `Healthy()` returns `true` are kept in `glb.healthyServers`. The request routing logic (`ChooseServer`) always selects from this filtered list, ensuring that unhealthy backends are automatically bypassed and traffic is never routed to failed services. ## Practical Configuration Example Here is a complete YAML configuration enabling HTTP health checks for a backend service: ```yaml pipeline: name: http-proxy kind: Proxy spec: serviceName: my-backend loadBalance: policy: roundRobin healthCheck: interval: 5s timeout: 2s fails: 2 passes: 1 uri: /healthz method: GET match: statusCodes: [[200, 299]] ``` When this pipeline starts: 1. The `LoadBalanceSpec` parses the `healthCheck` block into a `HealthCheckSpec`. 2. `NewHTTPHealthChecker` creates an HTTP health-checker with the specification. 3. `GeneralLoadBalancer.Init` registers the checker, starts the 5-second ticker, and begins probing `/healthz` on each backend. 4. Servers that fail two consecutive probes become unhealthy and are removed from the active pool; a single successful probe marks a server healthy again. ## Summary - **Easegress health checks** are declarative specifications defined in [`pkg/filters/proxies/healthcheck.go`](https://github.com/easegress-io/easegress/blob/main/pkg/filters/proxies/healthcheck.go) and extended by protocol-specific structs in [`pkg/filters/proxies/httpproxy/healthcheck.go`](https://github.com/easegress-io/easegress/blob/main/pkg/filters/proxies/httpproxy/healthcheck.go). - **HTTP and WebSocket checkers** implement the `HealthChecker` interface, executing probes using `http.Client` or `gorilla/websocket.Dialer` with configurable timeouts and match rules. - **Load balancer integration** occurs in [`pkg/filters/proxies/loadbalance.go`](https://github.com/easegress-io/easegress/blob/main/pkg/filters/proxies/loadbalance.go), where `GeneralLoadBalancer` runs periodic checks via `checkServers()`, updates health counters, and maintains a filtered pool of healthy backends. - **Automatic failover** ensures that only servers passing the configured `Passes` threshold are active, while servers reaching the `Fails` threshold are automatically removed from request routing. ## Frequently Asked Questions ### How do you configure HTTP health checks in Easegress? HTTP health checks are configured within the `loadBalance.healthCheck` section of a Proxy pipeline. You must specify the `interval`, `timeout`, `fails`, and `passes` thresholds, along with HTTP-specific fields like `uri`, `method`, and optionally `headers`, `body`, and `match` rules for response validation. The configuration is parsed into `HTTPHealthCheckSpec` in [`pkg/filters/proxies/httpproxy/healthcheck.go`](https://github.com/easegress-io/easegress/blob/main/pkg/filters/proxies/httpproxy/healthcheck.go). ### What is the difference between the 'fails' and 'passes' thresholds in Easegress health checks? The `fails` threshold defines how many consecutive unsuccessful probes must occur before a healthy server is marked as unhealthy and removed from the active pool. Conversely, the `passes` threshold specifies how many consecutive successful probes are required to promote an unhealthy server back to healthy status. These counters are maintained per-server in the `HealthCounter` field and evaluated in `GeneralLoadBalancer.checkServers()`. ### Does Easegress support WebSocket health checking? Yes, Easegress supports WebSocket health checks through the `WSProxyHealthCheckSpec` defined in [`pkg/filters/proxies/httpproxy/healthcheck.go`](https://github.com/easegress-io/easegress/blob/main/pkg/filters/proxies/httpproxy/healthcheck.go). This specification can include an optional HTTP health check to validate the base endpoint, followed by a WebSocket handshake validation using `gorilla/websocket.Dialer`. The checker validates the HTTP 101 Switching Protocols response to confirm the WebSocket upgrade capability. ### How does Easegress handle unhealthy backend servers? When a backend server reaches the configured `fails` threshold, the `GeneralLoadBalancer` sets the server's `Unhealth` flag to true and rebuilds the `healthyServers` pool, excluding the failed node. The `ChooseServer` method only selects from this filtered pool, ensuring traffic is automatically routed away from unhealthy backends. Once the server accumulates enough `passes` consecutive successes, it is reinstated into the active pool.