How API Rate Limits and Throttling Are Handled Across Macro Services

Macro uses a Redis-backed atomic sliding-window algorithm with Lua scripts to enforce consistent API rate limiting across Gmail, Google Calendar, and other microservices.

The macro-inc/macro platform unifies throttling policy through a shared RedisClient abstraction. Located in the email service utilities, this client ensures that external API calls—whether interactive user requests or bulk back-fill jobs—respect quota limits without scattering rate-limit logic throughout the codebase.

Core Rate-Limiting Architecture

The implementation centers on three components: a thin Redis connection wrapper, a typed argument struct, and an atomic Lua script that implements cost-aware sliding-window throttling.

RedisClient and RateLimitArgs

All rate-limit checks flow through RedisClient, which wraps a pooled async Redis connection. Consumers describe each request via RateLimitArgs:

pub struct RateLimitArgs {
    pub user_id: Uuid,
    pub operation: GmailApiOperation,
    pub is_backfill: bool,
}

• user_id: Identifies the tenant for per-user quota tracking
• operation: Enum variant determining the request cost (Gmail operations vary in expense)
• is_backfill: Flag that selects a reduced quota for bulk synchronization jobs

The Sliding-Window Lua Script

The method RedisClient::is_rate_limited executes a single atomic Lua script (get_rate_limit_script_with_usage) that:

1. Prunes entries older than the sliding window
2. Sums costs of remaining entries in the sorted set
3. Compares total against rate_limit_units (or rate_limit_units_backfill)
4. If allowed, inserts the new entry with microsecond-precision score and updates TTL

let (is_limited, _current_units): (i32, u32) = script
    .key(&redis_key)
    .arg(rate_limit_units)                 // quota limit
    .arg(self.rate_limit_secs * 1_000_000) // window size (μs)
    .arg(now_micros)                       // current timestamp
    .arg(cost)                             // cost of this operation
    .arg(&member_id)                       // unique member "cost:uuid"
    .invoke_async(&mut con)
    .await?;

Return values: 0 = allowed, 1 = denied. The script runs atomically at Redis, eliminating race conditions under concurrent load.

Service-Specific Implementations

Gmail API Throttling

Gmail operations use per-operation costs defined in the GmailApiOperation enum. In services/email_service/src/util/redis/rate_limit.rs, the is_rate_limited method:

• Constructs a per-user key: gmail-ratelimit:log:{user_id}
• Looks up operation-specific cost via operation.cost()
• Applies standard or back-fill quota based on is_backfill

Interactive users receive full quota; back-fill jobs receive rate_limit_units_backfill to prevent bulk sync from starving live traffic.

Google Calendar Throttling

Calendar API calls use a dedicated wrapper method is_calendar_rate_limited with simplified semantics:

• Fixed key pattern: calendar-ratelimit:log:{link_id}
• Fixed cost: 1 unit per request
• Quota: 300 queries per minute (50% of Google's published 600 qpm limit)

This conservative margin keeps calendar sync responsive while avoiding 429 errors from Google.

pub async fn is_calendar_rate_limited(&self, link_id: Uuid) -> bool {
    // Uses same Lua script with cost=1, limit=300/min
}

Cross-Service Reuse

While the implementation lives in email_service, other microservices reuse the same pattern:

• connection_gateway: Shares RedisClient for WebSocket connection throttling
• worker_trigger: Checks limits before launching ECS tasks for heavy operations
• notification_service: Applies Redis-backed back-pressure on outbound delivery

Lua Script Implementation Details

The script at get_rate_limit_script_with_usage implements true sliding-window behavior:

local key = KEYS[1]
local limit_units = tonumber(ARGV[1])
local window_micros = tonumber(ARGV[2])
local now_micros = tonumber(ARGV[3])
local new_request_units = tonumber(ARGV[4])
local new_member = ARGV[5]

local window_start = now_micros - window_micros
redis.call('ZREMRANGEBYSCORE', key, '-inf', window_start)  -- prune old

local members = redis.call('ZRANGE', key, 0, -1)
local current_units = 0
for _, member in ipairs(members) do
    local cost = tonumber(string.match(member, "^(%d+):"))
    if cost then current_units = current_units + cost end
end

if (current_units + new_request_units) > limit_units then
    return {1, current_units}            -- DENY
else
    redis.call('ZADD', key, now_micros, new_member)
    redis.call('EXPIRE', key, (window_micros / 1000000) * 2)
    return {0, current_units + new_request_units}  -- ALLOW
end

Key design decisions:

Decision	Rationale
Sorted set with microsecond scores	Enables precise sliding-window eviction
Member format cost:uuid	Encodes variable cost directly in entry
TTL = 2× window	Automatic cleanup without background jobs
Lua atomicity	No race conditions; single round-trip

Practical Usage Examples

Interactive Gmail Request

use crate::util::redis::{RedisClient, RateLimitArgs};
use models_email::gmail::operations::GmailApiOperation;

async fn send_message(
    redis: &RedisClient,
    user_id: Uuid,
) -> Result<Message, ApiError> {
    let args = RateLimitArgs {
        user_id,
        operation: GmailApiOperation::SendMessage, // cost = 5
        is_backfill: false,
    };

    if redis.is_rate_limited(args).await {
        return Err(ApiError::RateLimited);
    }

    // Proceed to Gmail API
    gmail_client.send_message(...).await
}

Bulk Back-Fill Job

async fn backfill_history(
    redis: &RedisClient,
    user_id: Uuid,
) -> Result<(), ApiError> {
    let args = RateLimitArgs {
        user_id,
        operation: GmailApiOperation::ListMessages, // cost = 2
        is_backfill: true, // Reduced quota path
    };

    if redis.is_rate_limited(args).await {
        tokio::time::sleep(Duration::from_secs(5)).await;
        return Ok(()); // Defer, don't fail
    }

    // Fetch and store history
    Ok(())
}

Calendar Sync Check

async fn sync_calendar(
    redis: &RedisClient,
    calendar_link_id: Uuid,
) -> Result<(), ApiError> {
    if redis.is_calendar_rate_limited(calendar_link_id).await {
        return Err(ApiError::CalendarQuotaExceeded);
    }

    calendar_client.list_events(...).await
}

Source File Reference

Component	Path
Rate-limit implementation	services/email_service/src/util/redis/rate_limit.rs
Gmail operation costs	services/email_service/src/models_email/gmail/operations.rs
RedisClient definition	services/email_service/src/util/redis/mod.rs
Back-fill handler example	services/email_sfs_delete_handler/src/handler.rs
Connection gateway usage	services/connection_gateway/src/service/redis.rs

Summary

• Unified RedisClient: Single abstraction for all services requiring API throttling
• Atomic Lua scripts: Sliding-window checks in one round-trip, no race conditions
• Cost-aware metering: Different Gmail operations consume different quota units
• Back-fill safety: Separate, lower quota prevents bulk jobs from impacting users
• Calendar-specific wrapper: Fixed-cost, conservative limits for Google Calendar
• Cross-service reuse: Pattern extends beyond email to gateway, notifications, and workers

Frequently Asked Questions

How does Macro prevent race conditions when checking rate limits?

The RedisClient executes all quota checks as atomic Lua scripts on Redis. Because Lua runs server-side as a single transaction, concurrent requests for the same user see a consistent state—no separate read-modify-write cycle that could allow quota overruns.

What happens when a rate limit is exceeded?

The is_rate_limited and is_calendar_rate_limited methods return true, signaling the caller to reject with 429 Too Many Requests, schedule a retry with exponential backoff, or defer the work to a later queue. The platform does not automatically queue blocked requests; services handle rejection policy appropriately.

Why use a sliding window instead of token or leaky bucket?

The sliding window provides smoother throttling behavior than fixed windows (which allow burst at boundaries) while remaining simpler to implement atomically in Redis than token bucket. The Lua script's cost-aware variant also accommodates operations of different expense without complex bucket state management.

Can rate limits be adjusted per tenant or environment?

Yes. The rate_limit_units, rate_limit_units_backfill, and rate_limit_secs values are configurable per RedisClient instance. Additionally, the is_backfill flag within RateLimitArgs provides runtime quota selection without requiring separate client instances.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}