# How to Configure Rate Limiting and Message Queuing for High-Traffic AstrBot Deployments

> Learn how to configure rate limiting and message queuing for high-traffic AstrBot deployments by adjusting cmd config and scaling WebChat queues for optimal performance.

- Repository: [AstrBot AI/AstrBot](https://github.com/AstrBotDevs/AstrBot)
- Tags: how-to-guide
- Published: 2026-03-12

---

**To configure rate limiting and message queuing for high-traffic AstrBot deployments, adjust the `platform_settings.rate_limit` parameters in [`data/cmd_config.json`](https://github.com/AstrBotDevs/AstrBot/blob/main/data/cmd_config.json) and scale the WebChat queue capacities via environment variables or runtime modifications to `webchat_queue_mgr`.**

AstrBot protects its platform from overload using two complementary mechanisms: **rate limiting** to throttle per-session message bursts and **message queuing** to serialize concurrent requests per conversation. According to the AstrBotDevs/AstrBot source code, both systems are controlled via configuration files and runtime parameters that require no code changes to tune for production traffic.

## Understanding AstrBot's Traffic Control Architecture

AstrBot implements a dual-layer protection system. The **rate limiter** enforces a fixed-window algorithm in [`astrbot/core/pipeline/rate_limit_check/stage.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/pipeline/rate_limit_check/stage.py) to cap messages per session, while the **message queue manager** in [`astrbot/core/platform/sources/webchat/webchat_queue_mgr.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/platform/sources/webchat/webchat_queue_mgr.py) buffers inbound and outbound traffic per conversation using `asyncio.Queue` objects.

The `RateLimitStage` class tracks per-session timestamps using a `defaultdict[deque[datetime]]` protected by `asyncio.Lock` instances to prevent race conditions. Meanwhile, the singleton `webchat_queue_mgr` maintains separate queues for incoming messages and back-queues for request/response pairing, ensuring sequential processing even under concurrent load.

## Configuring Rate Limiting for High-Traffic Scenarios

Rate limiting parameters reside in the global configuration under `platform_settings.rate_limit`. These values are read during the `initialize()` method of `RateLimitStage` (lines 31–41 in [`stage.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/stage.py)).

### Default Configuration Values

The default template in [`astrbot/core/config/default.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/config/default.py) defines the following baseline:

```json
{
  "platform_settings": {
    "rate_limit": {
      "time": 60,
      "count": 30,
      "strategy": "stall"
    }
  }
}

```

- **`time`**: The sliding window duration in seconds.
- **`count`**: Maximum allowed messages within that window per session.
- **`strategy`**: The enforcement behavior—`stall` delays processing until the next window, while `discard` aborts the current request via `event.stop_event()`.

### Tuning Strategies for Production Loads

Adjust these values in [`data/cmd_config.json`](https://github.com/AstrBotDevs/AstrBot/blob/main/data/cmd_config.json) (or via the Dashboard at **配置 → 平台设置**) to match your traffic patterns:

- **Allow higher bursts**: Increase `count` to `200` or `500` while maintaining `time` at `60` to accept more per-session throughput.
- **Reduce burst duration**: Decrease `time` to `30` seconds and proportionally raise `count` to provide more frequent reset points.
- **Prevent back-pressure**: Switch `strategy` to `discard` to drop overloaded sessions immediately, keeping the pipeline free for other users.
- **Enable graceful retries**: Keep `strategy` as `stall` but enlarge both `time` and `count` so clients experience short pauses rather than hard rejections.

**Example configuration for 1,000 messages per 30-second window with discard behavior:**

```json
{
  "platform_settings": {
    "rate_limit": {
      "time": 30,
      "count": 1000,
      "strategy": "discard"
    }
  }
}

```

The bot reloads these settings on restart or when triggered via the CLI command `astrbot cli reload-config`.

## Scaling Message Queues in WebChat

For the WebChat platform component, AstrBot uses per-conversation queues to serialize message processing. The singleton manager defined in [`webchat_queue_mgr.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/webchat_queue_mgr.py) controls these buffers.

### Default Queue Capacities

The `WebChatQueueMgr.__init__` method (lines 7–19) establishes these defaults:

- **`queue_maxsize`**: `128` — Maximum pending messages from the platform per conversation.
- **`back_queue_maxsize`**: `512` — Maximum pending responses for request/response pairing (used in streaming or tool call scenarios).

When queues reach capacity, subsequent `await queue.put()` calls raise `QueueFull`, causing immediate request failures.

### Runtime Configuration Methods

You can increase these limits without modifying source files using two approaches:

**Method 1: Environment Variables**

Export the variables before launching the bot:

```bash
export ASTRBOT_WEBCHAT_QUEUE_MAXSIZE=512
export ASTRBOT_WEBCHAT_BACK_QUEUE_MAXSIZE=2048
uv run main.py

```

**Method 2: Runtime Modification**

Override the singleton values at startup in your entry point:

```python
from astrbot.core.platform.sources.webchat.webchat_queue_mgr import webchat_queue_mgr

webchat_queue_mgr.queue_maxsize = 512
webchat_queue_mgr.back_queue_maxsize = 2048

```

These adjustments ensure bursty traffic does not block queue operations, absorbing spikes up to the new capacities before applying back-pressure.

## Production Deployment Checklist

For a high-traffic deployment capable of handling thousands of concurrent users:

1. **Set aggressive rate limits** in [`data/cmd_config.json`](https://github.com/AstrBotDevs/AstrBot/blob/main/data/cmd_config.json) using short windows and high counts (e.g., 30 seconds / 1,000 messages).
2. **Select appropriate strategy**: Use `discard` for strict overload protection or `stall` for graceful degradation.
3. **Scale WebChat queues**: Set `ASTRBOT_WEBCHAT_QUEUE_MAXSIZE` and `ASTRBOT_WEBCHAT_BACK_QUEUE_MAXSIZE` environment variables to `512` and `2048` respectively.
4. **Monitor logs**: Watch for `RateLimitStage` stall messages and queue-full warnings in [`astrbot/core/logger.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/logger.py) to fine-tune values.
5. **Verify reload behavior**: Confirm configuration changes take effect via restart or the admin API endpoint `/api/reload-config`.

## Summary

- **Rate limiting** is configured via `platform_settings.rate_limit` in [`data/cmd_config.json`](https://github.com/AstrBotDevs/AstrBot/blob/main/data/cmd_config.json), controlling window size (`time`), message cap (`count`), and enforcement strategy (`stall` or `discard`).
- The `RateLimitStage` class in [`astrbot/core/pipeline/rate_limit_check/stage.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/pipeline/rate_limit_check/stage.py) implements a fixed-window algorithm with per-session locking.
- **Message queuing** for WebChat is managed by the singleton `webchat_queue_mgr` in [`astrbot/core/platform/sources/webchat/webchat_queue_mgr.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/platform/sources/webchat/webchat_queue_mgr.py), using `asyncio.Queue` objects with configurable `queue_maxsize` and `back_queue_maxsize`.
- Queue capacities can be adjusted via environment variables (`ASTRBOT_WEBCHAT_QUEUE_MAXSIZE`, `ASTRBOT_WEBCHAT_BACK_QUEUE_MAXSIZE`) or runtime modification before the server starts.
- Changes to [`cmd_config.json`](https://github.com/AstrBotDevs/AstrBot/blob/main/cmd_config.json) require a restart or config reload to take effect.

## Frequently Asked Questions

### What is the difference between rate limiting and message queuing in AstrBot?

**Rate limiting** controls how many messages a single user session can send within a time window, protecting downstream services from individual user bursts. **Message queuing** buffers messages per conversation to ensure that concurrent requests are processed sequentially rather than interfering with each other. According to the AstrBotDevs/AstrBot source code, rate limiting is implemented in `RateLimitStage` while queuing is handled by `webchat_queue_mgr` for the WebChat platform.

### How does the `discard` strategy differ from `stall` in AstrBot rate limiting?

The `stall` strategy causes the pipeline to sleep until the rate limit window resets, creating back-pressure that delays the user but preserves the request. The `discard` strategy immediately aborts the current request using `event.stop_event()`, dropping the message entirely to keep the pipeline free for other traffic. Choose `discard` for strict overload protection and `stall` when you want graceful degradation with automatic client-side retry potential.

### Can I adjust WebChat queue sizes without restarting AstrBot?

No, the `queue_maxsize` and `back_queue_maxsize` parameters are initialized in `WebChatQueueMgr.__init__` and determine the `maxsize` parameter of `asyncio.Queue` objects at instantiation. To change these values, you must set the `ASTRBOT_WEBCHAT_QUEUE_MAXSIZE` and `ASTRBOT_WEBCHAT_BACK_QUEUE_MAXSIZE` environment variables before starting the process, or modify the singleton instance before the queues are created in your custom startup script.

### Where does AstrBot store the rate limit configuration?

AstrBot stores runtime configuration in [`data/cmd_config.json`](https://github.com/AstrBotDevs/AstrBot/blob/main/data/cmd_config.json), which is generated from the default template in [`astrbot/core/config/default.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/config/default.py). The rate limit settings reside under the `platform_settings.rate_limit` key. You can edit this file directly or use the Dashboard interface at **配置 → 平台设置** to adjust the `time`, `count`, and `strategy` parameters.