How to Troubleshoot Hook Backpressure and HTTP 429 Saturation in ai-memory

Hook requests return HTTP 429 when ai-memory's global semaphore (HookState.ingest_semaphore) or per-source token bucket (IngestRateLimiter) saturates, protecting the server from unbounded memory growth.

The ai-memory project by akitaonrails implements a protective backpressure system to prevent server overload during high-volume hook ingestion at the POST /hook and POST /hook/batch endpoints. When incoming requests exceed capacity limits, the system responds with HTTP 429 Too Many Requests to maintain stability. Understanding how to troubleshoot hook backpressure and HTTP 429 saturation in ai-memory requires examining both global concurrency controls and per-source rate limiting mechanisms implemented in the Rust source code.

Understanding the Backpressure Mechanisms

The ai-memory server employs two distinct layers of protection against overload. Each layer triggers specific log messages and metrics that help identify the root cause of saturation.

Global In-Flight Limit (Semaphore)

In crates/ai-memory-hooks/src/router.rs at lines 25-30, the server acquires a permit from HookState.ingest_semaphore before processing each hook event. The default DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT allows 1024 concurrent processing tasks. When the semaphore is exhausted, the server logs hook ingest saturated; dropping event with 429 and immediately returns HTTP 429 to the client without entering the processing queue.

Per-Source Rate Limiting (Token Bucket)

Lines 83-109 in router.rs implement a bounded token bucket via IngestRateLimiter. The limiter keys requests using ingest_rate_key (derived from session ID, user, and current working directory). With a default INGEST_RATE_MAX_KEYS of 4096 distinct sources, any source exhausting its token allocation triggers the warning hook ingest rate limit exceeded for source; dropping event with 429 and returns HTTP 429.

Metrics and Observability

Every acceptance, drop, and 429 event updates counters in HookState.ingest_metrics (ai_memory_core::IngestMetrics). The MCP /status endpoint exposes these as hook_ingest_accepted, hook_ingest_shed_saturated, and hook_ingest_shed_rate_limited, enabling real-time monitoring of saturation patterns.

Diagnosing HTTP 429 Saturation: Step-by-Step

When troubleshooting a flood of 429 responses, follow this diagnostic workflow to isolate whether the bottleneck is global concurrency, per-source throttling, or downstream writer saturation.

1. Examine Server Logs for Warning Patterns

Check the application logs for two specific warning messages emitted by router.rs. The message hook ingest saturated; dropping event with 429 indicates global semaphore exhaustion, while hook ingest rate limit exceeded for source; dropping event with 429 identifies per-source token bucket depletion. These distinct log entries immediately reveal which backpressure path is active.

2. Query the Ingest Metrics Endpoint

Send a GET request to the MCP /status endpoint to retrieve the JSON payload containing current counters. Inspect hook_ingest_shed_saturated versus hook_ingest_shed_rate_limited to quantify whether the 429s stem from global limits or individual source throttling.

// Example: querying the status endpoint to see current back-pressure metrics
use reqwest::Client;

#[tokio::main]
async fn main() {
    let client = Client::new();
    let status: serde_json::Value = client
        .get("http://127.0.0.1:49374/status")
        .send()
        .await
        .unwrap()
        .json()
        .await
        .unwrap();

    println!("Hook ingest metrics: {:#}", status["hook_ingest"]);
}

3. Verify Configured Limits Against Defaults

Compare your current configuration against the defaults defined in crates/ai-memory-cli/src/config.rs. The system defaults to 1024 max in-flight requests (DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT) and 4096 rate limiter keys (INGEST_RATE_MAX_KEYS). If your workload legitimately exceeds these values, increase them via environment variables.

4. Inspect Source Key Distribution

Analyze the ingest_rate_key generation logic in router.rs lines 98-115. A high cardinality of distinct keys (such as many short-lived agents with unique session IDs) can exhaust the LRU bucket capacity, causing thrashing. Consolidate agents under shared session tokens or increase INGEST_RATE_MAX_KEYS to accommodate your agent topology.

5. Tune the Token Bucket Parameters

Adjust the IngestRateLimiter refill rate and burst size if traffic patterns are bursty. Low refill_per_sec values cause rapid 429s during traffic spikes. Modify these via the CLI flags --ingest-rate-per-sec and --ingest-burst or their environment variable equivalents.

6. Scale the SQLite Writer Backend

The hook handler spawns asynchronous tasks that eventually write to the SQLite writer actor (crates/ai-memory-store/src/writer.rs). If the writer queue becomes a bottleneck, the upstream semaphore fills indirectly. Monitor WriterHandle metrics and increase writer threads using --writer-workers or migrate the database to faster SSD storage to alleviate downstream pressure.

7. Review Client Retry Behavior

Ensure clients implement exponential backoff when receiving 429 responses. Aggressive retries amplify saturation. The ai-memory hook capture CLI already implements this logic, but custom clients should respect Retry-After headers or use conservative backoff strategies.

// Example: client-side exponential back-off after receiving 429
async fn post_hook(event: &HookEvent) -> Result<(), reqwest::Error> {
    let mut delay = std::time::Duration::from_millis(100);
    for _ in 0..5 {
        let resp = client.post(&url).json(event).send().await?;
        if resp.status() != reqwest::StatusCode::TOO_MANY_REQUESTS {
            return resp.error_for_status(); // success
        }
        tokio::time::sleep(delay).await;
        delay *= 2; // exponential back‑off
    }
    Err(reqwest::Error::new(
        reqwest::ErrorKind::Request,
        "exhausted retries after 429",
    ))
}

8. Validate Batch Request Sizes

For batch ingestion via /hook/batch, verify that payloads do not exceed MAX_HOOK_BATCH_ITEMS = 256 items. Requests exceeding this limit receive HTTP 413 Payload Too Large rather than 429, but misconfigured clients may conflate these responses. Keeping batches under this threshold prevents unnecessary rejections and maintains low latency.

Configuration Tuning to Prevent 429 Errors

Adjust these parameters in ai_memory_cli::config to match your hardware capacity and traffic patterns:

• AI_MEMORY_INGEST_MAX_IN_FLIGHT (or --max-in-flight): Increase from 1024 to 4096 or higher when running on servers with sufficient CPU and memory.
• AI_MEMORY_INGEST_RATE_PER_SEC (or --ingest-rate-per-sec): Raise the token bucket refill rate to sustain higher sustained throughput.
• AI_MEMORY_INGEST_BURST (or --ingest-burst): Increase burst capacity to absorb traffic spikes without dropping events.
• **--writer-workers: Allocate additional threads to the SQLite writer to prevent downstream bottlenecks.

// Example: manually increasing the in‑flight limit via environment variable
// (run before starting the server)
std::env::set_var("AI_MEMORY_INGEST_MAX_IN_FLIGHT", "4096");

// Example: customizing the per‑source rate‑limiter from the CLI
// `ai-memory server --ingest-rate-per-sec 50 --ingest-burst 10`
use ai_memory_cli::config::CliConfig;

let cfg = CliConfig::parse(); // reads env vars / flags
let rate_limiter = IngestRateLimiter::new(cfg.ingest_rate_per_sec, cfg.ingest_burst);

Key Source Files for Deep Debugging

These files contain the implementation details necessary for advanced troubleshooting:

• crates/ai-memory-hooks/src/router.rs: Contains the HTTP handlers, semaphore acquisition at lines 25-30, rate-limiter logic at lines 83-109, and the warning logs that emit 429 responses.
• crates/ai-memory-core/src/ingest_metrics.rs: Defines the counters (record_shed_saturated, record_shed_rate_limited) exposed via the /status endpoint.
• crates/ai-memory-cli/src/config.rs: Parses environment variables and CLI flags that control the semaphore size and rate-limiter parameters.
• crates/ai-memory-mcp/src/server.rs: Hosts the MCP admin and status endpoints where you observe ingest metrics in production.
• crates/ai-memory-store/src/writer.rs: Implements the single-writer SQLite actor; its queue length directly influences hook semaphore pressure.

Summary

• Two mechanisms trigger HTTP 429: A global semaphore (DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT = 1024) and a per-source token bucket (INGESTRateLimiter with INGEST_RATE_MAX_KEYS = 4096).
• Identify the bottleneck: Check server logs for hook ingest saturated (global) versus hook ingest rate limit exceeded for source (per-source) messages.
• Monitor via /status: Query the MCP endpoint to view hook_ingest_shed_saturated and hook_ingest_shed_rate_limited counters.
• Tune configuration: Adjust AI_MEMORY_INGEST_MAX_IN_FLIGHT, AI_MEMORY_INGEST_RATE_PER_SEC, and --writer-workers to match hardware capacity.
• Client resilience: Implement exponential backoff in clients to avoid amplifying pressure during saturation events.

Frequently Asked Questions

What causes HTTP 429 errors in ai-memory?

HTTP 429 errors occur when the server activates backpressure protection. This happens in two scenarios: when the global semaphore (HookState.ingest_semaphore) reaches its DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT limit (1024 concurrent tasks), or when a specific source exhausts its token bucket allocation in the IngestRateLimiter. Both mechanisms prevent memory exhaustion and ensure the SQLite writer does not become overwhelmed.

How do I check if I'm hitting global or per-source limits?

Query the MCP /status endpoint and examine the hook_ingest JSON object. If hook_ingest_shed_saturated is incrementing, you are hitting the global semaphore limit. If hook_ingest_shed_rate_limited is rising, specific sources are exceeding their token bucket rates. Alternatively, check server logs for hook ingest saturated (global) versus hook ingest rate limit exceeded for source (per-source) warning messages.

What is the default concurrent hook limit?

The default global limit is 1024 concurrent in-flight hook processing tasks, defined by DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT in crates/ai-memory-hooks/src/router.rs. Additionally, the per-source rate limiter maintains a maximum of 4096 distinct keys (INGEST_RATE_MAX_KEYS) in its LRU cache. You can override these defaults using the AI_MEMORY_INGEST_MAX_IN_FLIGHT environment variable or the --max-in-flight CLI flag.

How should clients handle 429 responses?

Clients must implement exponential backoff with jitter when receiving HTTP 429 responses. The ai-memory hook capture CLI already includes this logic. Custom implementations should sleep for an initial duration (e.g., 100ms), double the delay after each consecutive 429, and respect the Retry-After header if present. Aggressive immediate retries will exacerbate server saturation and trigger additional 429 responses.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}