How FluentRead Manages Multiple Translation API Calls and Prevents Rate Limiting

FluentRead uses a centralized queue-based architecture with concurrency limits, intelligent retry logic, and local caching to manage multiple translation API calls while preventing rate limiting.

The open-source browser extension FluentRead (available at bistutu/fluentread) processes high volumes of translation requests when users browse multilingual content. To handle these requests without overwhelming translation service APIs, the codebase implements a sophisticated throttling and resilience system. This article examines how FluentRead manages multiple translation API calls and prevents rate limiting through its queue-based architecture.

Centralized Translation Architecture

The translateText Entry Point

Every translation request in FluentRead flows through a single entry point: translateText in entrypoints/utils/translateApi.ts. This function acts as the gatekeeper for all outgoing API calls, ensuring that no translation request bypasses the rate-limiting safeguards.

By centralizing requests through translateText, FluentRead maintains a consistent interface for caching, error handling, and queue management. The function accepts configuration options including maxRetries, retryDelay, and timeout, allowing callers to customize resilience behavior while maintaining system-wide concurrency limits.

Concurrency Control Through Queue Management

How the Translation Queue Works

FluentRead implements a concurrency-limited queue through enqueueTranslation in entrypoints/utils/translateQueue.ts. This module tracks active translations using two critical state variables: activeTranslations (current count of in-flight requests) and pendingTranslations (FIFO array of queued jobs).

When translateText receives a request, it delegates execution to enqueueTranslation. The queue checks canAcceptMoreTasks() against config.maxConcurrentTranslations (defaulting to 6 parallel requests). If the limit is reached, the job joins pendingTranslations. Each completed translation decrements activeTranslations and triggers processQueue() to start the next pending job.

Configuring Maximum Concurrent Requests

The concurrency limit is configurable through getMaxConcurrentTranslations() in the queue module, which respects settings defined in entrypoints/utils/config.ts and entrypoints/utils/model.ts. The default value of 6 parallel translations balances throughput with API rate limit safety, but users can adjust this based on their specific translation service quotas.

Resilience Patterns for Rate Limit Prevention

Retry Logic with Exponential Backoff

FluentRead prevents rate limit errors caused by transient failures through intelligent retry mechanisms. Inside translateText, the inner translationTask wraps the actual browser.runtime.sendMessage call in a retry loop with configurable parameters: maxRetries (default 3) and retryDelay (configurable delay between attempts).

This retry logic prevents immediate re-requests that could compound rate limiting issues. By spacing out retry attempts and limiting the total number of retries, FluentRead ensures that temporary API unavailability does not result in request storms that exhaust rate limits.

Timeout Handling

To prevent hung requests from occupying concurrency slots indefinitely, FluentRead implements strict timeout handling. The translationTask uses Promise.race to compete the API call against a timeout promise (default 45 seconds). If the timeout wins, the request fails and triggers the retry logic, freeing up the concurrency slot for other queued translations.

This timeout mechanism is critical for maintaining queue throughput. Without it, slow or stalled API responses could block the entire pipeline, reducing the effective concurrency limit and causing unnecessary queue buildup.

Local Caching Strategy

Before any request enters the translation queue, FluentRead checks for cached results using cache.localGet. Successful translations are stored via cache.localSet. This local caching layer bypasses the queue entirely for repeated translations, significantly reducing the total volume of API calls.

The cache implementation prevents redundant requests for identical text segments, which is particularly effective when users browse pages with repeated phrases or navigate back to previously translated content. By eliminating duplicate API calls, the cache serves as the first line of defense against rate limiting.

Implementation Examples

The following examples demonstrate how to interact with FluentRead's translation API while respecting rate limits:

import { translateText, cancelAllTranslations, getTranslationStatus } from '@/entrypoints/utils/translateApi';

// Basic translation with default rate limiting
async function translateContent() {
  const sourceText = 'Hello, world!';
  const result = await translateText(sourceText, document.title);
  console.log('Translation:', result);
}

// Custom retry configuration for unreliable networks
async function translateWithCustomRetry() {
  const result = await translateText('Technical documentation', 'Page Title', {
    maxRetries: 2,      // Reduce retries for faster failure
    retryDelay: 1500,   // 1.5 seconds between attempts
    timeout: 30000,     // 30 second timeout per request
  });
  return result;
}

// Monitor queue status to prevent UI overload
function checkTranslationStatus() {
  const status = getTranslationStatus();
  console.log(`Active: ${status.activeTranslations}, Pending: ${status.pendingTranslations}, Max: ${status.maxConcurrent}`);
}

// Cleanup when navigating away
function cleanupTranslations() {
  cancelAllTranslations();
}

Summary

FluentRead manages multiple translation API calls and prevents rate limiting through a multi-layered architecture:

• Centralized queue system (translateQueue.ts) enforces a default limit of 6 concurrent requests, queuing excess jobs in a FIFO buffer
• Intelligent retry logic with configurable delays and maximum attempts prevents request storms during transient failures
• Strict timeout handling (45 seconds default) ensures failed requests release concurrency slots promptly
• Local caching layer eliminates redundant API calls for repeated text segments
• Unified entry point (translateText) ensures all translation requests pass through these protective mechanisms

Frequently Asked Questions

How many translation requests can FluentRead process simultaneously?

FluentRead processes a maximum of 6 concurrent translation requests by default. This limit is configurable through config.maxConcurrentTranslations in the queue configuration. Additional requests are held in a FIFO queue (pendingTranslations) and processed as active slots become available.

What happens if a translation API request fails or times out?

Failed requests trigger the retry mechanism configured in translateText. By default, FluentRead retries failed requests up to 3 times with a configurable delay between attempts. If a request times out (default 45 seconds), it fails immediately and enters the retry loop. After exhausting retries, the error propagates to the caller.

Can I adjust the rate limiting settings for different translation services?

Yes, you can customize rate limiting behavior by modifying the configuration options passed to translateText or by adjusting the global config. Key parameters include maxRetries, retryDelay, and timeout. However, the maximum concurrent translations (maxConcurrentTranslations) is typically managed globally through the queue configuration to ensure system-wide stability.

Does FluentRead cache translation results to reduce API calls?

Yes, FluentRead implements a local caching layer that checks for existing translations before queueing new requests. The cache.localGet function checks for cached results, and successful translations are stored via cache.localSet. This caching mechanism significantly reduces API call volume, particularly for repeated text segments or when users revisit previously translated pages.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}