# What Metrics Does Easegress Expose? Complete Prometheus Monitoring Guide

> Explore Easegress Prometheus metrics for HTTP server health, latency, throughput, WAF, AI gateway, proxies, and file server cache. Monitor your API gateway performance effectively.

- Repository: [easegress-io/easegress](https://github.com/easegress-io/easegress)
- Tags: tutorial
- Published: 2026-03-01

---

**Easegress exposes comprehensive Prometheus metrics covering HTTP server health, request latency, throughput, WAF blocks, AI gateway token usage, proxy connection pools, and file server cache performance, all auto-registered and available at the `/metrics` endpoint.**

Easegress is a cloud-native traffic orchestration system that instruments its core components with detailed Prometheus metrics. The `easegress-io/easegress` repository defines counters, gauges, histograms, and summaries across HTTP servers, filters, and controllers in [`pkg/util/prometheushelper/helper.go`](https://github.com/easegress-io/easegress/blob/main/pkg/util/prometheushelper/helper.go), enabling deep observability into request processing, error rates, and resource utilization.

## HTTP Server Metrics

The HTTP server implementation in [`pkg/object/httpserver/runtime.go`](https://github.com/easegress-io/easegress/blob/main/pkg/object/httpserver/runtime.go) (lines 480-621) defines the most extensive metric collection in Easegress. These metrics track every aspect of request processing from health status to percentile latencies.

### Health and Availability

- **`httpserver_health`** – A **gauge** that reports `1` when the server is ready to accept traffic and `0` when it is down. Defined at line 480.

### Request and Response Counters

- **`httpserver_total_requests`** – Total incoming HTTP requests (line 484).
- **`httpserver_total_responses`** – Total HTTP responses sent (line 488).
- **`httpserver_total_error_requests`** – Requests resulting in 4xx or 5xx status codes (line 492).

### Latency and Size Distributions

- **`httpserver_requests_duration`** – **Histogram** measuring request-processing latency in milliseconds (line 496).
- **`httpserver_requests_size_bytes`** – **Histogram** tracking request body size in bytes (line 503).
- **`httpserver_responses_size_bytes`** – **Histogram** tracking response body size in bytes (line 510).

### Percentile Summaries

- **`httpserver_requests_duration_percentage`** – **Summary** providing percentile latency (p0-p100) (line 517).
- **`httpserver_requests_size_bytes_percentage`** – **Summary** for request-size percentiles (line 524).
- **`httpserver_responses_size_bytes_percentage`** – **Summary** for response-size percentiles (line 531).

### Rate and Error Gauges

Exponentially-weighted moving-average QPS metrics:
- **`httpserver_m1`**, **`httpserver_m5`**, **`httpserver_m15`** – QPS over 1, 5, and 15 minutes (lines 538-549).
- **`httpserver_m1_err`**, **`httpserver_m5_err`**, **`httpserver_m15_err`** – Error QPS over the same windows (lines 550-561).
- **`httpserver_m1_err_percent`**, **`httpserver_m5_err_percent`**, **`httpserver_m15_err_percent`** – Error-rate percentages (lines 562-573).

### Statistical Gauges

- **`httpserver_min`**, **`httpserver_max`**, **`httpserver_mean`** – Min, max, and mean latency in milliseconds (lines 574-586).
- **`httpserver_p25`**, **`httpserver_p50`**, **`httpserver_p75`**, **`httpserver_p95`**, **`httpserver_p98`**, **`httpserver_p99`**, **`httpserver_p999`** – Detailed percentile latencies (lines 586-610).
- **`httpserver_req_size`**, **`httpserver_resp_size`** – Total request and response byte counts in the current window (lines 614-621).

## Web Application Firewall (WAF) Metrics

The WAF controller in [`pkg/object/wafcontroller/metrics/metrics.go`](https://github.com/easegress-io/easegress/blob/main/pkg/object/wafcontroller/metrics/metrics.go) exposes security-specific counters.

- **`waf_total_refused_requests`** – **Counter** tracking the number of requests blocked by WAF rules (lines 81-86).

## AI Gateway Metrics

The AI gateway controller in [`pkg/object/aigatewaycontroller/metricshub/metricshub.go`](https://github.com/easegress-io/easegress/blob/main/pkg/object/aigatewaycontroller/metricshub/metricshub.go) (lines 131-163) provides LLM-specific observability.

### Request Tracking

- **`ai_gateway_total_request`** – **Counter** for all AI gateway requests (line 131).
- **`ai_gateway_success_request`** – **Counter** for successfully processed AI requests (line 136).
- **`ai_gateway_failed_request`** – **Counter** for failed requests, labeled with the **error** type (lines 141-145).

### Performance and Cost

- **`ai_gateway_requests_duration`** – **Histogram** measuring request latency for each AI provider (lines 146-152).
- **`ai_gateway_prompt_tokens`** – **Counter** for total prompt tokens processed (lines 154-158).
- **`ai_gateway_completion_tokens`** – **Counter** for total completion tokens generated (lines 159-163).

## Proxy Filter Metrics

The HTTP proxy filter in [`pkg/filters/proxies/httpproxy/pool.go`](https://github.com/easegress-io/easegress/blob/main/pkg/filters/proxies/httpproxy/pool.go) (lines 650-683) tracks connection pool performance and payload sizes.

### Connection Metrics

- **`proxy_total_connections`** – **Counter** for all connections handled by the proxy (line 650).
- **`proxy_total_error_connections`** – **Counter** for connections that resulted in an error (line 653).

### Payload Size Distributions

- **`proxy_request_body_size`** – **Histogram** for request body sizes in bytes (line 656).
- **`proxy_response_body_size`** – **Histogram** for response body sizes in bytes (line 663).

### Percentile Summaries

- **`proxy_request_body_size_percentage`** – **Summary** for request size percentiles (line 670).
- **`proxy_response_body_size_percentage`** – **Summary** for response size percentiles (line 677).

## File Server Metrics

The static file server filter in [`pkg/filters/fileserver/metrics.go`](https://github.com/easegress-io/easegress/blob/main/pkg/filters/fileserver/metrics.go) exposes cache performance metrics for buffer pools and memory-mapped files.

### Buffer Pool Metrics

- **`buffer_pool_hits`** – **Counter** for successful buffer-pool fetches (line 45).
- **`buffer_pool_miss`** – **Counter** for buffer-pool misses (line 50).
- **`buffer_pool_evicts`** – **Counter** for buffer-pool evictions (line 55).
- **`buffer_pool_ttl_evict`** – **Counter** for TTL-based evictions (line 60).
- **`buffer_pool_counter_size`** – **Gauge** for current number of buffers in the pool (line 65).
- **`buffer_pool_files`** – **Gauge** for number of files cached in the pool (line 70).

### Memory-Mapped File Metrics

- **`mmap_hits`** – **Counter** for successful memory-mapped file reads (line 81).
- **`mmap_miss`** – **Counter** for memory-map misses (line 86).
- **`mmap_files`** – **Gauge** for number of mmap-managed files (line 91).

## Accessing and Scraping Easegress Metrics

All metrics are **auto-registered** with the global Prometheus registry during component initialization. They are automatically exposed on the `/metrics` HTTP endpoint of the Easegress control plane, typically on port 9090.

### Basic Scraping with curl

Retrieve the raw Prometheus exposition format:

```bash
curl http://localhost:9090/metrics

```

The output includes HELP text, TYPE declarations, and labeled metric values:

```

# HELP httpserver_total_requests Total incoming HTTP requests

# TYPE httpserver_total_requests counter

httpserver_total_requests{clusterName="default",clusterRole="primary",instanceName="instance-001",httpServerName="demo-server",kind="httpserver",routerKind="...",backend="..."} 15420

```

### PromQL Query Examples

Calculate the 99th-percentile request latency for a specific HTTP server:

```promql
histogram_quantile(0.99, sum(rate(httpserver_requests_duration_bucket[5m])) by (le, httpServerName))

```

Monitor AI gateway token consumption rates:

```promql
sum(rate(ai_gateway_prompt_tokens[5m])) by (instanceName)

```

Track proxy error rates:

```promql
rate(proxy_total_error_connections[5m]) / rate(proxy_total_connections[5m])

```

### Programmatic Access in Go

To read metric values in unit tests or custom tooling, use the Prometheus client library with the helper utilities:

```go
import (
    "github.com/prometheus/client_golang/prometheus"
    "github.com/megaease/easegress/v2/pkg/util/prometheushelper"
)

// Create a gauge matching the HTTP server health metric
g := prometheushelper.NewGauge(
    "httpserver_health",
    "show the status for the http server: 1 for ready, 0 for down",
    []string{"clusterName", "clusterRole", "instanceName", "httpServerName", "kind"},
)

// Set values with specific labels
g.WithLabelValues("my-cluster", "master", "node-1", "my-http", "httpServer").Set(1)

// Register with a custom registry for isolation
reg := prometheus.NewRegistry()
reg.MustRegister(g)

```

### Prometheus Alerting Rules

Configure Alertmanager to detect HTTP server failures:

```yaml
groups:
- name: easegress-httpserver
  rules:
  - alert: HttpServerDown
    expr: httpserver_health == 0
    for: 1m
    labels:
      severity: critical
    annotations:
      summary: "HTTP server {{ $labels.httpServerName }} is down"
      description: "The HTTP server has reported a health value of 0 for more than 1 minute."

```

Alert on high error rates:

```yaml
  - alert: HighErrorRate
    expr: httpserver_m5_err_percent > 5
    for: 2m
    labels:
      severity: warning
    annotations:
      summary: "High error rate on {{ $labels.httpServerName }}"
      description: "Error rate is {{ $value }}% over the last 5 minutes."

```

## Summary

- Easegress instruments **HTTP servers**, **WAF controllers**, **AI gateways**, **proxy filters**, and **file servers** with Prometheus metrics using the `prometheushelper` utility package.
- The HTTP server implementation in [`pkg/object/httpserver/runtime.go`](https://github.com/easegress-io/easegress/blob/main/pkg/object/httpserver/runtime.go) provides the most comprehensive coverage, including latency histograms, percentile summaries, and moving-average QPS gauges.
- All metrics are **auto-registered** with the global Prometheus registry during component initialization and exposed on the `/metrics` endpoint without additional configuration.
- Metrics cover infrastructure health (`httpserver_health`), security blocks (`waf_total_refused_requests`), LLM costs (`ai_gateway_prompt_tokens`), and cache performance (`buffer_pool_hits`).
- You can scrape these metrics with standard Prometheus configurations, query them with PromQL, or access them programmatically using the Prometheus Go client library.

## Frequently Asked Questions

### How do I enable metrics in Easegress?

Metrics are **enabled by default** in Easegress. Each component automatically registers its metrics with the global Prometheus registry during initialization. You do not need to configure anything to expose the `/metrics` endpoint; it is available as soon as the Easegress instance starts on its configured admin port (typically 9090).

### What is the difference between histogram and summary metrics in Easegress?

**Histograms** (such as `httpserver_requests_duration`) bucket observations into configurable ranges and count events per bucket, making them ideal for aggregating latency distributions across multiple instances using `histogram_quantile()`. **Summaries** (such as `httpserver_requests_duration_percentage`) calculate configurable percentiles (p0-p100) using a sliding time window, providing exact quantiles for a specific instance but less suitable for aggregation across clusters.

### How can I create custom metrics for my Easegress filter?

Use the `prometheushelper` package located at [`pkg/util/prometheushelper/helper.go`](https://github.com/easegress-io/easegress/blob/main/pkg/util/prometheushelper/helper.go). Import the package and use wrapper functions like `prometheushelper.NewCounter()`, `prometheushelper.NewGauge()`, `prometheushelper.NewHistogram()`, or `prometheushelper.NewSummary()`. These functions automatically handle label naming and registration with the global registry, ensuring your custom metrics appear alongside built-in ones on the `/metrics` endpoint.

### Where are the AI gateway token metrics defined?

AI gateway token metrics are defined in [`pkg/object/aigatewaycontroller/metricshub/metricshub.go`](https://github.com/easegress-io/easegress/blob/main/pkg/object/aigatewaycontroller/metricshub/metricshub.go). Specifically, `ai_gateway_prompt_tokens` (lines 154-158) counts input tokens sent to LLM providers, while `ai_gateway_completion_tokens` (lines 159-163) counts output tokens generated. Both are counters that accumulate over time, allowing you to calculate token consumption rates using PromQL queries like `sum(rate(ai_gateway_prompt_tokens[5m]))`.