RocketMQ-Rust Message Ordering Guarantees: Per-Queue FIFO Explained

RocketMQ-Rust guarantees strict FIFO (first-in-first-out) ordering only within individual message queues, requiring either single-queue topics or hash-based queue selectors to maintain ordering across related messages.

Message ordering guarantees define whether consumers process events in the exact sequence producers sent them. In the mxsm/rocketmq-rust repository, the broker enforces FIFO semantics at the queue level through its commit log storage mechanism and sequential consumer fetch logic. Understanding these boundaries is essential for designing applications that require strict sequential processing versus those that can tolerate parallel consumption across multiple queues.

How FIFO Ordering Works in RocketMQ-Rust

The broker stores all messages in a commit log (rocketmq-store/src/log_file/commit_log.rs), where each MessageQueue represents an independent, append-only sequence. When producers publish to a specific queue, the broker writes entries sequentially to disk, and consumers retrieve them in that exact order via simple_consumer_impl.rs.

Critically, no ordering is enforced across different queues of the same topic. Messages routed to queue 0 and queue 1 may interleave arbitrarily during consumption, even if they share the same topic. This design prioritizes horizontal scalability over global ordering by default.

Strategies for Strict Message Ordering

To achieve ordering stronger than per-queue FIFO, you must control message routing explicitly. The codebase provides two primary mechanisms documented in the README (line ~249) and basic-concepts.md (lines ~138-140).

Single Queue Configuration for Global FIFO

The simplest approach to global ordering configures the topic with exactly one queue. All messages then flow through a single FIFO channel, eliminating interleaving entirely.

use rocketmq_client::producer::Producer;
use rocketmq_common::common::message::message_single::Message;

// Build a producer that sends to a topic configured with only one queue
let mut producer = Producer::builder()
    .topic("single_queue_topic")
    .build()
    .await?;

// Send messages; they will be consumed in the exact order sent
for i in 0..5 {
    let msg = Message::builder()
        .topic("single_queue_topic")
        .body(format!("msg-{}", i))
        .build()?;
    producer.send(msg).await?;
}

Because the broker stores these messages sequentially in the commit log without concurrent queue competition, consumers receive them in strict publication order.

Hash-Based Queue Selectors for Per-Key Ordering

For topics requiring multiple queues for throughput but ordering within logical groups (e.g., per-user or per-order), use the queue selector API. The SelectMessageQueueByHash implementation in rocketmq-client/src/producer/queue_selector/select_message_queue_by_hash.rs consistently maps hashable keys to specific queues.

use rocketmq_client::producer::{
    Producer, 
    queue_selector::{MessageQueueSelector, SelectMessageQueueByHash},
};
use rocketmq_common::common::message::message_single::Message;

// Create a producer that uses the hash selector
let selector = SelectMessageQueueByHash::new();
let mut producer = Producer::builder()
    .topic("ordered_topic")
    .queue_selector(selector)   // attach selector
    .build()
    .await?;

// All messages sharing the same order_id go to the same queue → FIFO per order_id
let order_id = 42u64; // any hashable key
for i in 0..5 {
    let msg = Message::builder()
        .topic("ordered_topic")
        .body(format!("order-{}-msg-{}", order_id, i))
        .build()?;
    // `send_by_selector` passes the key to the selector
    producer.send_by_selector(msg, order_id).await?;
}

The selector hashes order_id modulo the number of queues, ensuring all messages for that key route to the same physical queue while distinct keys distribute across queues for parallelism.

Consumer-Side Ordering Enforcement

Ordering guarantees rely on consumer behavior as implemented in rocketmq-client/src/consumer/consumer_impl/simple_consumer_impl.rs. The client fetches messages from each assigned queue sequentially, processing the commit log offset in ascending order without reordering.

use rocketmq_client::consumer::{Consumer, ConsumeMessage};

let mut consumer = Consumer::builder()
    .group("my_consumer_group")
    .topic("ordered_topic")
    .build()
    .await?;

consumer.start(|msg: ConsumeMessage| async move {
    // Messages are delivered in the order they were stored in the queue
    println!("Received: {:?}", msg.body_as_str());
    Ok(())
}).await?;

Because the consumer processes the queue as a sequential stream, it preserves the FIFO invariant established by the broker storage layer.

Implementation Details

The ordering semantics emerge from specific architectural components:

• Storage layer: rocketmq-store/src/log_file/commit_log.rs persists messages in append-only order, creating the physical FIFO sequence.
• Selector logic: rocketmq-client/src/producer/queue_selector/select_message_queue_by_hash.rs provides deterministic routing for partitioned ordering.
• Fetch logic: rocketmq-client/src/consumer/consumer_impl/simple_consumer_impl.rs reads messages sequentially from each queue without reordering.
• Documentation: The README and rocketmq-website/docs/getting-started/basic-concepts.md explicitly state that FIFO applies per-queue, not globally.

Summary

• Per-queue FIFO: RocketMQ-Rust guarantees strict ordering only within individual MessageQueue instances, enforced by the commit log append-only storage.
• No cross-queue ordering: Messages in different queues interleave arbitrarily; design your topology assuming queues are independent streams.
• Global ordering requires single queue: Configure topics with one queue or use a queue selector that maps all messages to the same queue.
• Partitioned ordering via hashing: Use SelectMessageQueueByHash to route related messages (e.g., same user ID) to consistent queues, achieving FIFO per logical key while maintaining parallelism across keys.
• Consumer cooperation: The consumer implementation respects queue ordering by fetching and processing messages sequentially from each assigned queue.

Frequently Asked Questions

Does RocketMQ-Rust guarantee global FIFO ordering across all queues?

No. According to the basic concepts documentation and source implementation, FIFO ordering applies strictly within a single message queue. When a topic contains multiple queues, messages may be consumed out of global order even if they were published sequentially, unless you explicitly route them to the same queue via selectors or single-queue configuration.

How do I maintain order for related messages like a specific user or order?

Use the hash-based queue selector provided in select_message_queue_by_hash.rs. By passing a consistent key (such as user_id or order_id) to send_by_selector(), the broker hashes that key to select the same queue for all related messages. This preserves FIFO ordering for that specific key while allowing unrelated keys to distribute across other queues.

What happens if I send messages without a queue selector to a multi-queue topic?

The producer uses a default round-robin or random selection strategy, distributing messages across all available queues. Consequently, related messages sent in sequence may land in different queues and be consumed concurrently or out of order. Without selector-based routing, you lose ordering guarantees beyond the individual queue level.

Where in the codebase is the FIFO guarantee physically enforced?

The guarantee originates in rocketmq-store/src/log_file/commit_log.rs, where messages are appended to disk in receive order. This physical sequence is maintained through simple_consumer_impl.rs, which fetches messages by offset without reordering. The queue selector logic in select_message_queue_by_hash.rs provides the routing mechanism to leverage this per-queue FIFO behavior for application-level ordering requirements.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}