How the Timeout Middleware Handles Long-Running API Requests in Omi

The Omi backend utilizes a custom TimeoutMiddleware to shield the service from stale connections and resource exhaustion by rejecting requests older than HTTP_MAXIMUM_AGE_SECONDS with a 408 status and terminating operations that exceed per-method time limits with a 504 Gateway Timeout.

The basedhardware/omi repository implements a defensive layer against hanging connections through a custom middleware defined in backend/utils/other/timeout.py. This mechanism intercepts every incoming request to validate its age and enforce strict execution limits. Understanding how the timeout middleware handles long-running API requests is essential for maintaining stability under load and preventing resource leaks.

Detecting Stale Requests Before Processing

The middleware first inspects incoming requests for the x-request-start-time header. This header should contain a Unix timestamp indicating when the client initiated the request.

The middleware converts this value and compares it against the current server time. If the elapsed duration exceeds HTTP_MAXIMUM_AGE_SECONDS (defaulting to 300 seconds or 5 minutes), the request is immediately rejected without executing any business logic.

# Conceptual flow from backend/utils/other/timeout.py

if elapsed_time > HTTP_MAXIMUM_AGE_SECONDS:
    return Response(status_code=408, content="Request is too old and has been rejected.")

This produces an immediate 408 Request Timeout response, protecting the backend from processing stale or delayed requests that may have already timed out on the client side.

Configuring Per-Method Timeout Limits

The middleware supports granular control through environment variables. The default timeout for all requests is governed by HTTP_DEFAULT_TIMEOUT (default 120 seconds or 2 minutes).

Individual HTTP methods can override this default via the methods_timeout dictionary. The implementation reads specific environment variables—HTTP_GET_TIMEOUT, HTTP_PUT_TIMEOUT, HTTP_PATCH_TIMEOUT, HTTP_DELETE_TIMEOUT, and HTTP_POST_TIMEOUT—and maps them to uppercase method names.

# Configuration pattern from backend/main.py

methods_timeout = {
    "GET":    os.getenv("HTTP_GET_TIMEOUT"),
    "PUT":    os.getenv("HTTP_PUT_TIMEOUT"),
    "PATCH":  os.getenv("HTTP_PATCH_TIMEOUT"),
    "DELETE": os.getenv("HTTP_DELETE_TIMEOUT"),
}

During request processing, the middleware selects the applicable timeout using:

timeout = self.methods_timeout.get(request.method, self.default_timeout)

Enforcing Timeouts at Runtime

Once the appropriate timeout duration is determined, the middleware wraps the route handler execution using asyncio.wait_for. The actual endpoint logic is invoked via call_next(request), which is passed to the asyncio wrapper along with the calculated timeout value.

# Core enforcement logic from backend/utils/other/timeout.py

return await asyncio.wait_for(call_next(request), timeout=timeout)

If the coroutine does not complete within the allotted timeframe, asyncio.TimeoutError is raised and caught by the middleware. The middleware then returns a 504 Gateway Timeout response, signaling that the upstream application failed to generate a timely response.

Integrating the Middleware into FastAPI

To activate protection, register TimeoutMiddleware with the FastAPI application in backend/main.py. Pass the methods_timeout mapping constructed from environment variables during application startup.

from utils.other.timeout import TimeoutMiddleware
import os

methods_timeout = {
    "GET":    os.getenv("HTTP_GET_TIMEOUT"),
    "PUT":    os.getenv("HTTP_PUT_TIMEOUT"),
    "PATCH":  os.getenv("HTTP_PATCH_TIMEOUT"),
    "DELETE": os.getenv("HTTP_DELETE_TIMEOUT"),
}

app = FastAPI()
app.add_middleware(TimeoutMiddleware, methods_timeout=methods_timeout)

Tuning Timeouts via Environment Variables

Configure the middleware behavior without code changes by setting these environment variables before starting the server:

export HTTP_DEFAULT_TIMEOUT=120      # Global default: 2 minutes

export HTTP_MAXIMUM_AGE_SECONDS=300  # Stale request threshold: 5 minutes

export HTTP_GET_TIMEOUT=30           # GET requests: 30 seconds

export HTTP_POST_TIMEOUT=180         # POST requests: 3 minutes

When a client includes the start time header, ensure it follows Unix timestamp format:

GET /api/v1/data HTTP/1.1
Host: api.example.com
x-request-start-time: 1709000000.0

If this request arrives more than HTTP_MAXIMUM_AGE_SECONDS after the specified timestamp, the server responds immediately with:

HTTP/1.1 408 Request Timeout
Content-Type: text/plain

Request is too old and has been rejected.

Summary

• Stale request filtering: The middleware checks the x-request-start-time header and rejects requests exceeding HTTP_MAXIMUM_AGE_SECONDS with a 408 Request Timeout.
• Method-specific limits: Configure timeouts per HTTP method using environment variables like HTTP_GET_TIMEOUT and HTTP_POST_TIMEOUT, stored in the methods_timeout dictionary.
• Asyncio enforcement: Route handlers execute within asyncio.wait_for, which raises asyncio.TimeoutError if the limit is exceeded, triggering a 504 Gateway Timeout response.
• FastAPI integration: The middleware is registered in backend/main.py and implemented in backend/utils/other/timeout.py.

Frequently Asked Questions

What status code does Omi return for stale requests?

If a request arrives with an x-request-start-time header indicating it is older than the configured HTTP_MAXIMUM_AGE_SECONDS (default 5 minutes), the middleware immediately returns 408 Request Timeout without processing the request body.

How do I set different timeout limits for specific HTTP methods?

Set environment variables prefixed with HTTP_ and suffixed with the method name, such as HTTP_GET_TIMEOUT=30 or HTTP_POST_TIMEOUT=180. The middleware loads these into the methods_timeout dictionary and applies them based on the request method, falling back to HTTP_DEFAULT_TIMEOUT for unspecified methods.

What happens when an API endpoint exceeds its timeout limit?

The middleware wraps the endpoint execution in asyncio.wait_for. If the operation does not complete within the calculated timeout, an asyncio.TimeoutError is caught and the middleware returns 504 Gateway Timeout to indicate the server failed to produce a timely response.

Where is the TimeoutMiddleware defined in the Omi codebase?

The implementation resides in backend/utils/other/timeout.py, while the registration and configuration logic appears in backend/main.py where the middleware is added to the FastAPI application instance using app.add_middleware(TimeoutMiddleware, ...).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}