# How to Configure Rate Limiting for API Calls Made by Ax

> Easily configure rate limiting for API calls made by Ax. Ax's token-based rate limiter tracks usage and delays requests to manage your token budget effectively.

- Repository: [Ax/ax](https://github.com/ax-llm/ax)
- Tags: how-to-guide
- Published: 2026-02-25

---

**Ax provides a built-in token-based rate limiter that tracks API usage and automatically delays requests until your configured token budget replenishes.**

The Ax library includes sophisticated rate limiting capabilities designed to prevent quota exhaustion when calling LLM providers like Groq, Mistral, and X-Grok. By configuring rate limiting for API calls made by Ax, you can enforce token-per-minute budgets or implement completely custom throttling strategies that match your specific infrastructure constraints.

## Understanding Ax's Token-Based Rate Limiter

At the core of Ax's rate limiting system is the **`AxRateLimiterTokenUsage`** class, implemented in **[src/ax/util/rate-limit.ts](https://github.com/ax-llm/ax/blob/main/src/ax/util/rate-limit.ts)**. This class implements a token bucket algorithm that manages two critical parameters:

- **Token budget**: The maximum number of tokens allowed per time window (typically per minute)
- **Refill rate**: How quickly tokens regenerate (`tokensPerMinute / 60`)

When you configure an AI provider in Ax, the constructor automatically instantiates this limiter unless you provide a custom alternative.

## Configuring Rate Limits on AI Providers

Ax exposes rate limiting configuration through the `options` parameter when constructing any AI provider. The system checks for a custom `rateLimiter` function first, then falls back to creating an `AxRateLimiterTokenUsage` instance using the `tokensPerMinute` value.

### Setting a Custom Token Budget

The simplest way to configure rate limiting is to set the **`tokensPerMinute`** option when creating a provider. For example, when using Groq (which defaults to 4800 tokens per minute), you can override this budget:

```typescript
import { AxAIGroq } from '@ax-llm/ax';

const groq = new AxAIGroq({
  apiKey: process.env.GROQ_APIKEY!,
  // Limit to 3,000 tokens per minute
  options: { tokensPerMinute: 3000 },
});

```

In **[src/ax/ai/groq/api.ts](https://github.com/ax-llm/ax/blob/main/src/ax/ai/groq/api.ts)** (lines 118-124), the constructor passes this value to `new AxRateLimiterTokenUsage(tokensPerMin, tokensPerMin / 60)`, creating a limiter that refills at 50 tokens per second when configured for 3,000 tokens per minute.

### Implementing a Custom Rate Limiter Function

For advanced use cases—such as request-count-based limiting, external quota systems, or multi-tenant throttling—you can provide a custom **`rateLimiter`** function via the provider options:

```typescript
import { AxAIGroq, type AxRateLimiterFunction } from '@ax-llm/ax';

let callsInCurrentMinute = 0;
let minuteStart = Date.now();

const requestCountLimiter: AxRateLimiterFunction = async (fn, info) => {
  const now = Date.now();
  
  // Reset counter every minute
  if (now - minuteStart >= 60_000) {
    minuteStart = now;
    callsInCurrentMinute = 0;
  }
  
  // Enforce max 30 calls per minute
  if (callsInCurrentMinute >= 30) {
    const waitTime = minuteStart + 60_000 - now;
    await new Promise((r) => setTimeout(r, waitTime));
    minuteStart = Date.now();
    callsInCurrentMinute = 0;
  }
  
  callsInCurrentMinute++;
  return fn();
};

const groq = new AxAIGroq({
  apiKey: process.env.GROQ_APIKEY!,
  options: { rateLimiter: requestCountLimiter },
});

```

The provider invokes this function before every LLM request, passing the original request function and metadata about the call. Your custom logic can inspect `info.modelUsage?.tokens?.totalTokens` to make decisions based on anticipated token consumption.

### Disabling Rate Limiting

While not recommended for production, you can disable rate limiting by providing a no-op function:

```typescript
const noopLimiter: AxRateLimiterFunction = async (fn) => fn();

const groq = new AxAIGroq({
  apiKey: process.env.GROQ_APIKEY!,
  options: { rateLimiter: noopLimiter },
});

```

Alternatively, setting `tokensPerMinute` to an extremely high value effectively disables throttling while maintaining the token-tracking infrastructure.

## Practical Code Examples

### Basic Token Budget Configuration

This example configures a Groq provider with a conservative 2,000 tokens per minute limit, suitable for applications with strict quota constraints:

```typescript
import { AxAIGroq } from '@ax-llm/ax';

const groq = new AxAIGroq({
  apiKey: process.env.GROQ_APIKEY!,
  options: { tokensPerMinute: 2000 },
});

await groq.chat({ 
  messages: [{ role: 'user', content: 'Explain rate limiting strategies' }] 
});

```

As implemented in **[src/ax/ai/groq/api.ts](https://github.com/ax-llm/ax/blob/main/src/ax/ai/groq/api.ts)**, this configuration creates an `AxRateLimiterTokenUsage` instance that refills approximately 33 tokens per second.

### Stand-Alone Rate Limiter Usage

For scenarios where you need to protect non-Ax API calls or implement custom batching logic, you can use `AxRateLimiterTokenUsage` directly:

```typescript
import { AxRateLimiterTokenUsage } from '@ax-llm/ax/util';

const limiter = new AxRateLimiterTokenUsage(
  5000,        // tokensPerMinute (bucket size)
  5000 / 60,   // refillRate (tokens per second)
  { debug: true }
);

async function protectedApiCall(fn: () => Promise<any>, tokenCount: number) {
  await limiter.acquire(tokenCount);
  return fn();
}

// Usage: protect a function that consumes 120 tokens
await protectedCall(() => fetchExpensiveData(), 120);

```

The `acquire()` method, defined in **[src/ax/util/rate-limit.ts](https://github.com/ax-llm/ax/blob/main/src/ax/util/rate-limit.ts)**, handles the token bucket logic, including waiting for sufficient tokens to become available before resolving.

## Summary

- **Ax implements token-based rate limiting** through the `AxRateLimiterTokenUsage` class in [`src/ax/util/rate-limit.ts`](https://github.com/ax-llm/ax/blob/main/src/ax/util/rate-limit.ts), using a token bucket algorithm that tracks consumption and refill rates.
- **Configure limits via `tokensPerMinute`** when constructing AI providers like `AxAIGroq` to automatically create a token budget that refills at `tokensPerMinute / 60` tokens per second.
- **Implement custom strategies** by providing a `rateLimiter` function in the provider options, enabling request-count limits, external quota integration, or multi-tenant throttling.
- **Reference implementations** in [`src/ax/ai/groq/api.ts`](https://github.com/ax-llm/ax/blob/main/src/ax/ai/groq/api.ts), [`src/ax/ai/mistral/api.ts`](https://github.com/ax-llm/ax/blob/main/src/ax/ai/mistral/api.ts), and [`src/ax/ai/x-grok/api.ts`](https://github.com/ax-llm/ax/blob/main/src/ax/ai/x-grok/api.ts) demonstrate how providers instantiate limiters and expose configuration options.

## Frequently Asked Questions

### How does Ax calculate the token refill rate when using tokensPerMinute?

Ax divides the `tokensPerMinute` value by 60 to determine the refill rate in tokens per second. For example, if you configure `tokensPerMinute: 3000`, the limiter refills at 50 tokens per second. This calculation occurs in the provider constructor, such as in [`src/ax/ai/groq/api.ts`](https://github.com/ax-llm/ax/blob/main/src/ax/ai/groq/api.ts), where the value is passed to `new AxRateLimiterTokenUsage(tokensPerMin, tokensPerMin / 60)`.

### Can I use Ax's rate limiter for non-Ax API calls?

Yes, you can instantiate `AxRateLimiterTokenUsage` directly from [`src/ax/util/rate-limit.ts`](https://github.com/ax-llm/ax/blob/main/src/ax/util/rate-limit.ts) and use it to protect any asynchronous function. Create an instance with your desired token budget and refill rate, then call `await limiter.acquire(tokenCount)` before executing your function. This approach works for external APIs, database queries, or any resource you want to throttle based on token consumption.

### What happens if I don't configure tokensPerMinute or a custom rateLimiter?

If you do not specify either option, Ax uses provider-specific defaults. For example, the Groq provider defaults to 4800 tokens per minute as defined in [`src/ax/ai/groq/api.ts`](https://github.com/ax-llm/ax/blob/main/src/ax/ai/groq/api.ts). The constructor automatically creates an `AxRateLimiterTokenUsage` instance with this default value, ensuring that all API calls are throttled according to the token bucket algorithm even without explicit configuration.

### How do I implement a request-count-based limit instead of token-based?

Pass a custom `rateLimiter` function to the provider options that tracks the number of requests rather than tokens. Your function should accept `fn` (the original request) and `info` (call metadata), implement your counting logic (such as incrementing a counter and checking against a maximum), and return `fn()` once conditions are met. This approach allows you to enforce limits like "30 requests per minute" regardless of token consumption per request.