How Easegress Handles Request and Response Caching: Route and Memory Cache Explained

Easegress implements two distinct caching layers—route caching at the HTTP server level using an ARC cache for routing decisions, and response caching in the HTTP proxy filter using an in-memory store for full HTTP responses—to optimize performance and reduce backend load.

The open-source Easegress project (megaease/easegress) provides high-performance traffic management through these complementary caching mechanisms. Understanding how Easegress handles request and response caching is essential for optimizing gateway performance and reducing latency in production deployments.

Route Caching in the HTTP Server

Route caching accelerates request processing by storing routing decisions before they reach the backend pipeline. This layer caches which pipeline or backend should handle a specific request based on the host, method, and path combination.

How Route Caching Works

When a request arrives at the HTTP server, the muxInstance.getRouteFromCache method in pkg/object/httpserver/mux.go constructs a cache key by concatenating the request's Host, Method, and Path using stringtool.Cat:

// pkg/object/httpserver/mux.go
func (mi *muxInstance) getRouteFromCache(req *httpprot.Request) *cachedRoute {
    if mi.cache != nil {
        key := stringtool.Cat(req.Host(), req.Method(), req.Path())
        if value, ok := mi.cache.Get(key); ok {
            return value.(*cachedRoute)
        }
    }
    return nil
}

If a cached entry exists, the router immediately returns the stored cachedRoute, bypassing the full rule-matching algorithm. After a successful route lookup, the result is stored using putRouteToCache:

// pkg/object/httpserver/mux.go
func (mi *muxInstance) putRouteToCache(req *httpprot.Request, rc *cachedRoute) {
    if mi.cache != nil {
        key := stringtool.Cat(req.Host(), req.Method(), req.Path())
        mi.cache.Add(key, rc)
    }
}

Configuration and Implementation Details

The route cache uses an ARC (Adaptive Replacement Cache) implementation from hashicorp/golang-lru. You enable it by setting spec.cacheSize to a value greater than zero in your HTTP server configuration:

# config/httpserver.yaml

apiVersion: v2
kind: HTTPServer
metadata:
  name: my-http
spec:
  address: ":80"
  cacheSize: 2048        # Enable route caching with up to 2048 entries

  rules:
    - host: "example.com"
      paths:
        - path: "/api"
          backend: my-backend

The cache is initialized during server reload in pkg/object/httpserver/mux.go:

// pkg/object/httpserver/mux.go
if spec.CacheSize > 0 {
    arc, err := lru.NewARC(int(spec.CacheSize))
    // ...
    inst.cache = arc
}

Response Caching in the HTTP Proxy

While route caching stores routing decisions, MemoryCache stores full HTTP responses to serve subsequent identical requests without hitting the backend. This implementation resides in pkg/filters/proxies/httpproxy/memorycache.go and uses patrickmn/go-cache for in-memory storage.

Cache Key Generation and Lookup

The cache key combines scheme, host, path, and method:

// pkg/filters/proxies/httpproxy/memorycache.go
func (mc *MemoryCache) key(req *httpprot.Request) string {
    return stringtool.Cat(req.Scheme(), req.Host(), req.Path(), req.Method())
}

When processing a request, the Load method checks multiple conditions before returning a cached entry:

• The request method must be listed in spec.methods
• The Cache-Control request header must not contain no-cache
• A valid entry must exist for the computed key

// pkg/filters/proxies/httpproxy/memorycache.go
func (mc *MemoryCache) Load(req *httpprot.Request) *CacheEntry {
    // method check …
    // request Cache‑Control check …
    if v, ok := mc.cache.Get(mc.key(req)); ok {
        return v.(*CacheEntry)
    }
    return nil
}

Storage Logic and HTTP Semantics

The Store method enforces strict HTTP caching semantics before persisting a response:

• Status code validation: Only codes listed in spec.codes are cached
• Size limits: Responses exceeding spec.maxEntryBytes are rejected
• Cache-Control directives: no-store, no-cache, or must-revalidate prevent caching
• CORS restrictions: Responses with specific Access-Control-Allow-Origin values (non-wildcard) are excluded

// pkg/filters/proxies/httpproxy/memorycache.go
func (mc *MemoryCache) Store(req *httpprot.Request, resp *httpprot.Response) {
    // stream, size, method, status‑code, CORS, request/response Cache‑Control checks …
    key := mc.key(req)
    entry := &CacheEntry{
        StatusCode: resp.StatusCode(),
        Header:     resp.HTTPHeader().Clone(),
        Body:       resp.RawPayload(),
    }
    mc.cache.SetDefault(key, entry)   // expiration comes from spec.Expiration
}

Configuration Example

Enable response caching by adding a memoryCache block to your proxy filter configuration:

# config/pipeline.yaml

apiVersion: v2
kind: Pipeline
metadata:
  name: cached-proxy
spec:
  filters:
    - name: proxy
      kind: Proxy
      pools:
        - name: api-pool
          servers:
            - url: http://api-service:8080
      memoryCache:
        expiration: 1m
        maxEntryBytes: 524288   # 512 KB

        codes: [200]
        methods: ["GET"]

Semantic Cache for AI Gateway

For AI-specific workloads, Easegress provides SemanticCache in pkg/object/aigatewaycontroller/middlewares/semanticcache.go. Unlike the exact-match MemoryCache, this layer stores vector embeddings and response data in external vector databases (PostgreSQL, Redis) to enable semantic reuse of AI completions based on embedding similarity rather than exact URL matching.

Summary

• Route caching accelerates request routing by storing host+method+path to pipeline mappings in an ARC cache within the HTTP server (pkg/object/httpserver/mux.go).
• Response caching stores complete HTTP responses in an in-memory cache (pkg/filters/proxies/httpproxy/memorycache.go) with strict adherence to HTTP semantics including Cache-Control headers, status code filtering, and CORS restrictions.
• Both caches are configured via YAML specifications—cacheSize for routing and the memoryCache block for responses—allowing fine-grained control over expiration, size limits, and eligible HTTP methods.

Frequently Asked Questions

How do I enable route caching in Easegress?

Set the cacheSize field to a positive integer in your HTTPServer specification. This activates the ARC cache in pkg/object/httpserver/mux.go, which stores up to that many routing decisions based on host, method, and path combinations.

What is the difference between route caching and response caching in Easegress?

Route caching stores only the routing decision—which pipeline or backend should handle a request—while response caching stores the complete HTTP response including status code, headers, and body. Route caching operates at the HTTP server level before request processing, whereas response caching operates within the proxy filter after receiving the backend response.

How does Easegress handle Cache-Control headers when caching responses?

The MemoryCache implementation in pkg/filters/proxies/httpproxy/memorycache.go respects both request and response Cache-Control directives. It refuses to store responses containing no-store, no-cache, or must-revalidate headers, and it bypasses the cache when the request includes no-cache. This ensures compliance with HTTP caching semantics while maximizing cache hit rates for cacheable content.

Can I use semantic caching for non-AI workloads in Easegress?

The SemanticCache in pkg/object/aigatewaycontroller/middlewares/semanticcache.go is specifically designed for AI gateway use cases and relies on vector embeddings stored in external vector databases. For general HTTP caching, you should use the MemoryCache in the HTTP proxy filter, which provides exact-match caching suitable for standard REST API responses.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}