Read Frog Translation Queue Configuration: Capacity, Rate Limiting, and Batching Parameters

The Read Frog extension exposes ten configurable parameters across two specialized queues—a token-bucket rate-limited request queue and a character-aware batch queue—allowing runtime tuning of throughput, burst capacity, and retry behavior via background message handlers.

The open-source Read Frog browser extension (available at mengxi-ream/read-frog) implements a sophisticated dual-queue architecture to manage LLM translation requests efficiently. Understanding the translation queue configuration options is essential for optimizing API costs and preventing rate-limit errors when processing high volumes of text.

Request Queue Parameters (Rate Limiting and Capacity)

The Request Queue utilizes a token-bucket algorithm implemented in src/utils/request/request-queue.ts to govern individual LLM API calls.

Token-Bucket Configuration Options

Five parameters control the rate-limited request flow:

• rate: Tokens added per second, effectively defining requests per second.
• capacity: Maximum tokens the bucket can hold, determining burst capacity.
• timeoutMs: Maximum duration a request may run before rejection.
• maxRetries: Number of retry attempts performed on failure.
• baseRetryDelayMs: Base backoff delay in milliseconds for exponential retry logic.

The queue is instantiated in src/entrypoints/background/translation-queues.ts with the following defaults:

const requestQueue = new RequestQueue({
  rate,
  capacity,
  timeoutMs: 20_000,
  maxRetries: 2,
  baseRetryDelayMs: 1_000,
});

Runtime Updates via Message Handlers

The background script listens for setTranslateRequestQueueConfig messages to update parameters without restarting the extension. According to the source in translation-queues.ts:

onMessage("setTranslateRequestQueueConfig", (msg) => {
  requestQueue.setQueueOptions(msg.data);
});

The setQueueOptions method validates incoming data against requestQueueConfigSchema and merges it with existing configuration using deepmerge.

Batch Queue Parameters (Batching Behavior)

The Batch Queue (src/utils/request/batch-queue.ts) groups multiple translation requests into single LLM calls to improve throughput and reduce costs.

Batch Size and Timing Controls

Three parameters define batch accumulation limits:

• maxCharactersPerBatch: Upper limit on total characters per batch call.
• maxItemsPerBatch: Maximum number of translation items grouped together.
• batchDelay: Minimum wait time in milliseconds before flushing a partial batch.

Fallback and Retry Resilience

Two additional parameters handle failure scenarios:

• maxRetries: Retry attempts before abandoning a batch.
• enableFallbackToIndividual: Boolean flag that, when true, automatically retries failed batches as individual requests through the Request Queue.

Instantiation in translation-queues.ts demonstrates these defaults:

const batchQueue = new BatchQueue<TranslateBatchData, string>({
  maxCharactersPerBatch,
  maxItemsPerBatch,
  batchDelay: 100,
  maxRetries: 3,
  enableFallbackToIndividual: true,
  getBatchKey: (data) => Sha256Hex(`${data.langConfig.sourceCode}-${data.langConfig.targetCode}-${data.providerConfig.id}`),
  getCharacters: (data) => data.text.length,
  // executeBatch and executeIndividual handlers omitted
});

Dynamic Batch Configuration

Similar to the request queue, the batch queue accepts runtime updates via the setTranslateBatchQueueConfig message handler:

onMessage("setTranslateBatchQueueConfig", (msg) => {
  batchQueue.setBatchConfig(msg.data);
});

The setBatchConfig method validates against batchQueueConfigSchema before applying changes.

Practical Configuration Examples

Adjusting Rate Limits at Runtime

Send a message from any content script or popup to modify throughput:

chrome.runtime.sendMessage({
  type: "setTranslateRequestQueueConfig",
  data: {
    rate: 5,          // 5 requests per second
    capacity: 10,     // allow bursts of up to 10 requests
    timeoutMs: 30000, // longer timeout for slower LLMs
    maxRetries: 3,
    baseRetryDelayMs: 2000,
  },
});

Tuning Batch Processing

Optimize for different content types by adjusting character limits and delays:

chrome.runtime.sendMessage({
  type: "setTranslateBatchQueueConfig",
  data: {
    maxCharactersPerBatch: 4000, // keep each batch under 4 KB
    maxItemsPerBatch: 8,         // at most 8 translations per batch
    batchDelay: 200,             // wait up to 200 ms before flushing
  },
});

Manual Queue Instantiation

For testing or custom implementations, instantiate queues directly:

import { RequestQueue } from "@/utils/request/request-queue";
import { BatchQueue } from "@/utils/request/batch-queue";

const rq = new RequestQueue({
  rate: 3,
  capacity: 6,
  timeoutMs: 15000,
  maxRetries: 1,
  baseRetryDelayMs: 500,
});

const bq = new BatchQueue({
  maxCharactersPerBatch: 3000,
  maxItemsPerBatch: 5,
  batchDelay: 100,
  maxRetries: 2,
  enableFallbackToIndividual: true,
  getBatchKey: (d) => d.providerConfig.id,
  getCharacters: (d) => d.text.length,
  executeBatch: async (list) => {/* call LLM with concatenated texts */},
  executeIndividual: async (item) => {/* call LLM for single item */},
});

Summary

• Read Frog implements two distinct queues: a Request Queue for rate-limited individual API calls and a Batch Queue for grouped translations.
• Request Queue parameters include rate, capacity, timeoutMs, maxRetries, and baseRetryDelayMs, configured in src/utils/request/request-queue.ts.
• Batch Queue parameters include maxCharactersPerBatch, maxItemsPerBatch, batchDelay, maxRetries, and enableFallbackToIndividual, defined in src/utils/request/batch-queue.ts.
• Both queues support runtime reconfiguration via background message handlers (setTranslateRequestQueueConfig and setTranslateBatchQueueConfig) without requiring extension restarts.
• Configuration changes are validated against Zod schemas (requestQueueConfigSchema and batchQueueConfigSchema) before application.

Frequently Asked Questions

What is the default request rate limit in Read Frog?

The default values are instantiated in src/entrypoints/background/translation-queues.ts with a timeoutMs of 20,000ms, maxRetries of 2, and baseRetryDelayMs of 1,000ms. The actual rate and capacity values are read from the user's configuration file at startup, allowing custom initial limits.

How does the batch queue handle failed translations?

When a batch request fails, the queue first retries according to maxRetries. If all retries exhaust and enableFallbackToIndividual is true (the default), the batch is automatically split and retried as individual requests through the Request Queue. This fallback mechanism is implemented in src/utils/request/batch-queue.ts to maximize success rates while minimizing API costs.

Can I disable batching and translate items individually?

While there is no single "disable batching" flag, setting maxItemsPerBatch to 1 effectively forces individual translation requests. Alternatively, setting enableFallbackToIndividual to false prevents automatic fallback to individual requests, though this increases the risk of total batch failure.

Where are the queue configurations stored and validated?

Type definitions reside in src/types/config/translate.ts, which exports RequestQueueConfig and BatchQueueConfig interfaces along with Zod validation schemas. Runtime updates are processed in src/entrypoints/background/translation-queues.ts, where setQueueOptions and setBatchConfig methods merge and validate changes using deepmerge and schema validation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}