# How Easegress Handles Request and Response Caching: Route and Memory Cache Explained

> Discover how Easegress optimizes performance with two caching layers: route caching for decisions and in-memory response caching for full HTTP responses. Reduce backend load efficiently.

- Repository: [MegaEase/easegress](https://github.com/megaease/easegress)
- Tags: deep-dive
- Published: 2026-03-07

---

**Easegress implements two distinct caching layers—route caching at the HTTP server level using an ARC cache for routing decisions, and response caching in the HTTP proxy filter using an in-memory store for full HTTP responses—to optimize performance and reduce backend load.**

The open-source Easegress project (`megaease/easegress`) provides high-performance traffic management through these complementary caching mechanisms. Understanding how Easegress handles request and response caching is essential for optimizing gateway performance and reducing latency in production deployments.

## Route Caching in the HTTP Server

Route caching accelerates request processing by storing routing decisions before they reach the backend pipeline. This layer caches which pipeline or backend should handle a specific request based on the host, method, and path combination.

### How Route Caching Works

When a request arrives at the HTTP server, the `muxInstance.getRouteFromCache` method in [`pkg/object/httpserver/mux.go`](https://github.com/megaease/easegress/blob/main/pkg/object/httpserver/mux.go) constructs a cache key by concatenating the request's **Host**, **Method**, and **Path** using `stringtool.Cat`:

```go
// pkg/object/httpserver/mux.go
func (mi *muxInstance) getRouteFromCache(req *httpprot.Request) *cachedRoute {
    if mi.cache != nil {
        key := stringtool.Cat(req.Host(), req.Method(), req.Path())
        if value, ok := mi.cache.Get(key); ok {
            return value.(*cachedRoute)
        }
    }
    return nil
}

```

If a cached entry exists, the router immediately returns the stored `cachedRoute`, bypassing the full rule-matching algorithm. After a successful route lookup, the result is stored using `putRouteToCache`:

```go
// pkg/object/httpserver/mux.go
func (mi *muxInstance) putRouteToCache(req *httpprot.Request, rc *cachedRoute) {
    if mi.cache != nil {
        key := stringtool.Cat(req.Host(), req.Method(), req.Path())
        mi.cache.Add(key, rc)
    }
}

```

### Configuration and Implementation Details

The route cache uses an **ARC** (Adaptive Replacement Cache) implementation from `hashicorp/golang-lru`. You enable it by setting `spec.cacheSize` to a value greater than zero in your HTTP server configuration:

```yaml

# config/httpserver.yaml

apiVersion: v2
kind: HTTPServer
metadata:
  name: my-http
spec:
  address: ":80"
  cacheSize: 2048        # Enable route caching with up to 2048 entries

  rules:
    - host: "example.com"
      paths:
        - path: "/api"
          backend: my-backend

```

The cache is initialized during server reload in [`pkg/object/httpserver/mux.go`](https://github.com/megaease/easegress/blob/main/pkg/object/httpserver/mux.go):

```go
// pkg/object/httpserver/mux.go
if spec.CacheSize > 0 {
    arc, err := lru.NewARC(int(spec.CacheSize))
    // ...
    inst.cache = arc
}

```

## Response Caching in the HTTP Proxy

While route caching stores routing decisions, **MemoryCache** stores full HTTP responses to serve subsequent identical requests without hitting the backend. This implementation resides in [`pkg/filters/proxies/httpproxy/memorycache.go`](https://github.com/megaease/easegress/blob/main/pkg/filters/proxies/httpproxy/memorycache.go) and uses `patrickmn/go-cache` for in-memory storage.

### Cache Key Generation and Lookup

The cache key combines **scheme**, **host**, **path**, and **method**:

```go
// pkg/filters/proxies/httpproxy/memorycache.go
func (mc *MemoryCache) key(req *httpprot.Request) string {
    return stringtool.Cat(req.Scheme(), req.Host(), req.Path(), req.Method())
}

```

When processing a request, the `Load` method checks multiple conditions before returning a cached entry:

- The request method must be listed in `spec.methods`
- The `Cache-Control` request header must not contain `no-cache`
- A valid entry must exist for the computed key

```go
// pkg/filters/proxies/httpproxy/memorycache.go
func (mc *MemoryCache) Load(req *httpprot.Request) *CacheEntry {
    // method check …
    // request Cache‑Control check …
    if v, ok := mc.cache.Get(mc.key(req)); ok {
        return v.(*CacheEntry)
    }
    return nil
}

```

### Storage Logic and HTTP Semantics

The `Store` method enforces strict HTTP caching semantics before persisting a response:

- **Status code validation**: Only codes listed in `spec.codes` are cached
- **Size limits**: Responses exceeding `spec.maxEntryBytes` are rejected
- **Cache-Control directives**: `no-store`, `no-cache`, or `must-revalidate` prevent caching
- **CORS restrictions**: Responses with specific `Access-Control-Allow-Origin` values (non-wildcard) are excluded

```go
// pkg/filters/proxies/httpproxy/memorycache.go
func (mc *MemoryCache) Store(req *httpprot.Request, resp *httpprot.Response) {
    // stream, size, method, status‑code, CORS, request/response Cache‑Control checks …
    key := mc.key(req)
    entry := &CacheEntry{
        StatusCode: resp.StatusCode(),
        Header:     resp.HTTPHeader().Clone(),
        Body:       resp.RawPayload(),
    }
    mc.cache.SetDefault(key, entry)   // expiration comes from spec.Expiration
}

```

### Configuration Example

Enable response caching by adding a `memoryCache` block to your proxy filter configuration:

```yaml

# config/pipeline.yaml

apiVersion: v2
kind: Pipeline
metadata:
  name: cached-proxy
spec:
  filters:
    - name: proxy
      kind: Proxy
      pools:
        - name: api-pool
          servers:
            - url: http://api-service:8080
      memoryCache:
        expiration: 1m
        maxEntryBytes: 524288   # 512 KB

        codes: [200]
        methods: ["GET"]

```

## Semantic Cache for AI Gateway

For AI-specific workloads, Easegress provides **SemanticCache** in [`pkg/object/aigatewaycontroller/middlewares/semanticcache.go`](https://github.com/megaease/easegress/blob/main/pkg/object/aigatewaycontroller/middlewares/semanticcache.go). Unlike the exact-match MemoryCache, this layer stores vector embeddings and response data in external vector databases (PostgreSQL, Redis) to enable semantic reuse of AI completions based on embedding similarity rather than exact URL matching.

## Summary

- **Route caching** accelerates request routing by storing `host+method+path` to pipeline mappings in an ARC cache within the HTTP server ([`pkg/object/httpserver/mux.go`](https://github.com/megaease/easegress/blob/main/pkg/object/httpserver/mux.go)).
- **Response caching** stores complete HTTP responses in an in-memory cache ([`pkg/filters/proxies/httpproxy/memorycache.go`](https://github.com/megaease/easegress/blob/main/pkg/filters/proxies/httpproxy/memorycache.go)) with strict adherence to HTTP semantics including Cache-Control headers, status code filtering, and CORS restrictions.
- Both caches are configured via YAML specifications—`cacheSize` for routing and the `memoryCache` block for responses—allowing fine-grained control over expiration, size limits, and eligible HTTP methods.

## Frequently Asked Questions

### How do I enable route caching in Easegress?

Set the `cacheSize` field to a positive integer in your HTTPServer specification. This activates the ARC cache in [`pkg/object/httpserver/mux.go`](https://github.com/megaease/easegress/blob/main/pkg/object/httpserver/mux.go), which stores up to that many routing decisions based on host, method, and path combinations.

### What is the difference between route caching and response caching in Easegress?

Route caching stores only the routing decision—which pipeline or backend should handle a request—while response caching stores the complete HTTP response including status code, headers, and body. Route caching operates at the HTTP server level before request processing, whereas response caching operates within the proxy filter after receiving the backend response.

### How does Easegress handle Cache-Control headers when caching responses?

The MemoryCache implementation in [`pkg/filters/proxies/httpproxy/memorycache.go`](https://github.com/megaease/easegress/blob/main/pkg/filters/proxies/httpproxy/memorycache.go) respects both request and response Cache-Control directives. It refuses to store responses containing `no-store`, `no-cache`, or `must-revalidate` headers, and it bypasses the cache when the request includes `no-cache`. This ensures compliance with HTTP caching semantics while maximizing cache hit rates for cacheable content.

### Can I use semantic caching for non-AI workloads in Easegress?

The SemanticCache in [`pkg/object/aigatewaycontroller/middlewares/semanticcache.go`](https://github.com/megaease/easegress/blob/main/pkg/object/aigatewaycontroller/middlewares/semanticcache.go) is specifically designed for AI gateway use cases and relies on vector embeddings stored in external vector databases. For general HTTP caching, you should use the MemoryCache in the HTTP proxy filter, which provides exact-match caching suitable for standard REST API responses.