how-to-guide

How the MCP /hook Endpoint Handles Rate Limiting and Backpressure in ai-memory

August 25, 2026 akitaonrails/ai-memory ↗

The MCP /hook endpoint in ai-memory immediately returns 202 Accepted for all requests and processes lifecycle events asynchronously, using an IngestRateLimiter to reject excessive bursts with 429 Too Many Requests and a tokio::sync::Semaphore to cap concurrent background tasks, preventing CPU and memory exhaustion under load.

The ai-memory repository implements a Memory-Control-Protocol (MCP) server that ingests lifecycle-hook events via the HTTP /hook path. Because these events are fire-and-forget by design, the server must balance high throughput with system stability, employing complementary rate limiting and backpressure mechanisms to remain responsive during traffic spikes.

Ingestion Rate Limiting with IngestRateLimiter

Per-client burst protection is enforced by the IngestRateLimiter type defined in crates/ai-memory-hooks/src/ingest.rs. This component tracks submission rates and caps the number of hook events a single client may submit within a short time window.

When a POST request arrives at /hook, the handler checks the rate limiter before queuing work. If the client exceeds the configured burst size, the endpoint returns 429 Too Many Requests immediately, protecting downstream components from overload. When rate limiting is not required, the server instantiates the limiter via ai_memory_hooks::IngestRateLimiter::disabled(), allowing unrestricted ingestion.

// Simplified handler logic from crates/ai-memory-mcp/src/server.rs
async fn handle_hook(req: Request<Body>) -> Result<Response<Body>, McpError> {
    // Sanitize payload (lines 998-1002)
    let payload = sanitize_payload(req.body()).await?;
    
    // Check per-client rate limit
    if !ingest_rate_limiter.allow(&payload.client_id) {
        return Ok(Response::builder()
            .status(StatusCode::TOO_MANY_REQUESTS)
            .body(Body::empty())?);
    }
    
    // Queue async work and return immediately
    tokio::spawn(process_hook(payload));
    Ok(Response::builder()
        .status(StatusCode::ACCEPTED)
        .body(Body::empty())?)
}

Backpressure Control via Semaphore

To prevent an unbounded number of background tasks from overwhelming the system, the MCP server utilizes a tokio::sync::Semaphore named rerank_gate in crates/ai-memory-mcp/src/server.rs. Although originally intended for rerank operations, this Arc<Semaphore> is reused to gate hook-related background work such as merging observations and updating the active project.

The semaphore initializes with RERANK_MAX_IN_FLIGHT permits. When processing a hook, the server attempts to acquire a permit via try_acquire(). If the semaphore is exhausted, the server falls back to the buffered path, returning 202 Accepted without waiting for the extra work, effectively shedding load rather than blocking.

// Backpressure implementation excerpt from server.rs
let rerank_gate = Arc::new(tokio::sync::Semaphore::new(RERANK_MAX_IN_FLIGHT));

// When starting heavy hook work:
let permit = match rerank_gate.try_acquire() {
    Ok(p) => p,
    Err(_) => {
        tracing::debug!("Back‑pressure: dropping extra hook work");
        return; // Return early, preserving system resources
    }
};

// Perform work...
drop(permit); // Release slot for next request

Asynchronous Processing and Active Project Updates

The /hook endpoint maintains a low-latency contract by never waiting for database writes or state updates. After passing the rate limiter check, the handler spawns an asynchronous task via tokio::spawn to handle the heavy lifting, including updating the active_project state (lines 60-66) and persisting observations to the store.

This separation ensures that clients receive a 202 Accepted response within milliseconds, while the server manages actual processing behind the semaphore gate. The background task eventually completes the work without blocking new inbound requests.

Client Activity Buffering

To mitigate database write storms, the server maintains in-memory buffers for per-client tool call counts. The flush_client_activity_loop (lines 90-124 in server.rs) flushes these counters in batches once per minute rather than writing individual records for every hook invocation. This throttling mechanism prevents a flood of tiny DB writes that would otherwise result from high-frequency hook calls.

Stress Testing and Validation

The test suite in crates/ai-memory-mcp/tests/ validates these protective mechanisms under load:

  • autoscope_stress.rs fires bursts of concurrent /hook requests and asserts that each receives a 202 response, confirming non-blocking behavior under stress (see the fire_hook_and_settle loop and the comment "burst hook … must accept under load").
  • handoff_admission.rs verifies that webhooks can be configured as blocking (synchronous execution) or non-blocking (skipped when under backpressure), demonstrating how the server uses backpressure flags to control hook execution paths.

Summary

  • Immediate response: The endpoint always returns 202 Accepted before processing, ensuring low latency for clients.
  • Burst protection: IngestRateLimiter rejects excessive requests with 429, preventing individual clients from monopolizing resources.
  • Concurrency caps: The rerank_gate semaphore limits concurrent background tasks, applying hard backpressure when the system is saturated.
  • Write batching: flush_client_activity_loop aggregates client activity to avoid database write storms.
  • Validated resilience: Stress tests in autoscope_stress.rs confirm stable behavior under extreme load.

Frequently Asked Questions

What HTTP status codes does the MCP /hook endpoint return?

The endpoint returns 202 Accepted for successfully ingested requests, indicating the payload has been queued for asynchronous processing. If the IngestRateLimiter detects a burst violation, it returns 429 Too Many Requests to signal that the client must back off.

How does ai-memory prevent database overload from hook events?

The server implements client activity buffering via flush_client_activity_loop in server.rs, which aggregates per-client tool call counts in memory and flushes them to the database once per minute. This batching strategy prevents the "write storm" that would occur if every hook call triggered an immediate database write.

Where is the rate limiting logic implemented in the codebase?

The rate limiting policy is defined in crates/ai-memory-hooks/src/ingest.rs within the IngestRateLimiter type. The MCP server in crates/ai-memory-mcp/src/server.rs instantiates this limiter and checks it inside the /hook handler before queuing background work.

Can hook processing be configured to run synchronously?

Yes. According to the test suite in handoff_admission.rs, webhooks support a blocking configuration flag. When enabled, the hook must run synchronously; when disabled (non-blocking), the server may skip the work under backpressure, illustrating how the semaphore gate controls execution semantics.

Have a question about this repo?

These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:

Share the following with your agent to get started:
curl -s "https://instagit.com/install.md"
Works with
Claude Codex Cursor VS Code OpenClaw Any MCP Client

Maintain an open-source project? Get it listed too →