# How the Timeout Middleware Handles Long-Running API Requests in Omi

> Learn how Omi's Timeout Middleware guards your API against slow requests. It rejects stale connections with 408 and terminates long operations with 504 errors.

- Repository: [omi/omi](https://github.com/basedhardware/omi)
- Tags: internals
- Published: 2026-02-26

---

**The Omi backend utilizes a custom `TimeoutMiddleware` to shield the service from stale connections and resource exhaustion by rejecting requests older than `HTTP_MAXIMUM_AGE_SECONDS` with a 408 status and terminating operations that exceed per-method time limits with a 504 Gateway Timeout.**

The `basedhardware/omi` repository implements a defensive layer against hanging connections through a custom middleware defined in [`backend/utils/other/timeout.py`](https://github.com/basedhardware/omi/blob/main/backend/utils/other/timeout.py). This mechanism intercepts every incoming request to validate its age and enforce strict execution limits. Understanding how the timeout middleware handles long-running API requests is essential for maintaining stability under load and preventing resource leaks.

## Detecting Stale Requests Before Processing

The middleware first inspects incoming requests for the **`x-request-start-time`** header. This header should contain a Unix timestamp indicating when the client initiated the request.

The middleware converts this value and compares it against the current server time. If the elapsed duration exceeds **`HTTP_MAXIMUM_AGE_SECONDS`** (defaulting to 300 seconds or 5 minutes), the request is immediately rejected without executing any business logic.

```python

# Conceptual flow from backend/utils/other/timeout.py

if elapsed_time > HTTP_MAXIMUM_AGE_SECONDS:
    return Response(status_code=408, content="Request is too old and has been rejected.")

```

This produces an immediate **408 Request Timeout** response, protecting the backend from processing stale or delayed requests that may have already timed out on the client side.

## Configuring Per-Method Timeout Limits

The middleware supports granular control through environment variables. The default timeout for all requests is governed by **`HTTP_DEFAULT_TIMEOUT`** (default 120 seconds or 2 minutes).

Individual HTTP methods can override this default via the `methods_timeout` dictionary. The implementation reads specific environment variables—**`HTTP_GET_TIMEOUT`**, **`HTTP_PUT_TIMEOUT`**, **`HTTP_PATCH_TIMEOUT`**, **`HTTP_DELETE_TIMEOUT`**, and **`HTTP_POST_TIMEOUT`**—and maps them to uppercase method names.

```python

# Configuration pattern from backend/main.py

methods_timeout = {
    "GET":    os.getenv("HTTP_GET_TIMEOUT"),
    "PUT":    os.getenv("HTTP_PUT_TIMEOUT"),
    "PATCH":  os.getenv("HTTP_PATCH_TIMEOUT"),
    "DELETE": os.getenv("HTTP_DELETE_TIMEOUT"),
}

```

During request processing, the middleware selects the applicable timeout using:

```python
timeout = self.methods_timeout.get(request.method, self.default_timeout)

```

## Enforcing Timeouts at Runtime

Once the appropriate timeout duration is determined, the middleware wraps the route handler execution using **`asyncio.wait_for`**. The actual endpoint logic is invoked via `call_next(request)`, which is passed to the asyncio wrapper along with the calculated timeout value.

```python

# Core enforcement logic from backend/utils/other/timeout.py

return await asyncio.wait_for(call_next(request), timeout=timeout)

```

If the coroutine does not complete within the allotted timeframe, **`asyncio.TimeoutError`** is raised and caught by the middleware. The middleware then returns a **504 Gateway Timeout** response, signaling that the upstream application failed to generate a timely response.

## Integrating the Middleware into FastAPI

To activate protection, register `TimeoutMiddleware` with the FastAPI application in [`backend/main.py`](https://github.com/basedhardware/omi/blob/main/backend/main.py). Pass the `methods_timeout` mapping constructed from environment variables during application startup.

```python
from utils.other.timeout import TimeoutMiddleware
import os

methods_timeout = {
    "GET":    os.getenv("HTTP_GET_TIMEOUT"),
    "PUT":    os.getenv("HTTP_PUT_TIMEOUT"),
    "PATCH":  os.getenv("HTTP_PATCH_TIMEOUT"),
    "DELETE": os.getenv("HTTP_DELETE_TIMEOUT"),
}

app = FastAPI()
app.add_middleware(TimeoutMiddleware, methods_timeout=methods_timeout)

```

## Tuning Timeouts via Environment Variables

Configure the middleware behavior without code changes by setting these environment variables before starting the server:

```bash
export HTTP_DEFAULT_TIMEOUT=120      # Global default: 2 minutes

export HTTP_MAXIMUM_AGE_SECONDS=300  # Stale request threshold: 5 minutes

export HTTP_GET_TIMEOUT=30           # GET requests: 30 seconds

export HTTP_POST_TIMEOUT=180         # POST requests: 3 minutes

```

When a client includes the start time header, ensure it follows Unix timestamp format:

```http
GET /api/v1/data HTTP/1.1
Host: api.example.com
x-request-start-time: 1709000000.0

```

If this request arrives more than `HTTP_MAXIMUM_AGE_SECONDS` after the specified timestamp, the server responds immediately with:

```http
HTTP/1.1 408 Request Timeout
Content-Type: text/plain

Request is too old and has been rejected.

```

## Summary

- **Stale request filtering**: The middleware checks the `x-request-start-time` header and rejects requests exceeding `HTTP_MAXIMUM_AGE_SECONDS` with a **408 Request Timeout**.
- **Method-specific limits**: Configure timeouts per HTTP method using environment variables like `HTTP_GET_TIMEOUT` and `HTTP_POST_TIMEOUT`, stored in the `methods_timeout` dictionary.
- **Asyncio enforcement**: Route handlers execute within `asyncio.wait_for`, which raises `asyncio.TimeoutError` if the limit is exceeded, triggering a **504 Gateway Timeout** response.
- **FastAPI integration**: The middleware is registered in [`backend/main.py`](https://github.com/basedhardware/omi/blob/main/backend/main.py) and implemented in [`backend/utils/other/timeout.py`](https://github.com/basedhardware/omi/blob/main/backend/utils/other/timeout.py).

## Frequently Asked Questions

### What status code does Omi return for stale requests?

If a request arrives with an `x-request-start-time` header indicating it is older than the configured `HTTP_MAXIMUM_AGE_SECONDS` (default 5 minutes), the middleware immediately returns **408 Request Timeout** without processing the request body.

### How do I set different timeout limits for specific HTTP methods?

Set environment variables prefixed with `HTTP_` and suffixed with the method name, such as `HTTP_GET_TIMEOUT=30` or `HTTP_POST_TIMEOUT=180`. The middleware loads these into the `methods_timeout` dictionary and applies them based on the request method, falling back to `HTTP_DEFAULT_TIMEOUT` for unspecified methods.

### What happens when an API endpoint exceeds its timeout limit?

The middleware wraps the endpoint execution in `asyncio.wait_for`. If the operation does not complete within the calculated timeout, an `asyncio.TimeoutError` is caught and the middleware returns **504 Gateway Timeout** to indicate the server failed to produce a timely response.

### Where is the TimeoutMiddleware defined in the Omi codebase?

The implementation resides in **[`backend/utils/other/timeout.py`](https://github.com/basedhardware/omi/blob/main/backend/utils/other/timeout.py)**, while the registration and configuration logic appears in **[`backend/main.py`](https://github.com/basedhardware/omi/blob/main/backend/main.py)** where the middleware is added to the FastAPI application instance using `app.add_middleware(TimeoutMiddleware, ...)`.