How OpenMAIC Implements Generation Retry Logic: A Code-Deep Analysis

OpenMAIC implements generation retry logic through a centralized withGenerationRetry helper in packages/@openmaic/generation/src/generation-retry.ts, which wraps async operations with configurable exponential back-off, intelligent error classification, and result-based retry predicates.

The THU-MAIC/OpenMAIC repository provides a sophisticated, declarative approach to handling transient failures in LLM-powered generation workflows. Rather than scattering retry logic across dozens of call sites, the codebase centralizes resilience patterns in a single, well-typed utility that every generation flow—from slides to quizzes to interactive widgets—can leverage consistently.

Core Architecture of the Retry System

The generation retry logic in OpenMAIC rests on six integrated components, all implemented in the same source file.

GenerationRetryOptions: Configuration Interface

The GenerationRetryOptions<T> interface exposes fine-grained control over retry behavior. Parameters include:

• label – Human-readable identifier for logging
• maxRetries – Maximum retry attempts (default varies by caller)
• baseDelayMs / maxDelayMs – Back-off timing bounds
• signal – AbortController integration
• sleep – Injectable sleep function for testability
• random – Jitter source (default: Math.random)
• shouldRetryResult – Predicate to trigger retry based on returned value
• onRetry – Callback for progress surfacing (UI notifications, logs)

isRetryableGenerationError: Error Classification Engine

This function performs deep inspection of errors to determine retry eligibility. As implemented in generation-retry.ts, it evaluates:

Check	Implementation
Explicit flag	error.isRetryable === true
HTTP status codes	Retryable: 408, 409, 425, 429; Non-retryable: 400-404, 422
Network indicators	Substring matches: rate limit, ECONNRESET, ETIMEDOUT, etc.
Timeout detection	error.name === 'TimeoutError' or similar
Nested error traversal	Recursively inspects cause, lastError, errors fields

This recursive design handles wrapped errors from HTTP clients, LLM SDKs, and intermediate processors without flattening useful context.

retryDelayMs: Exponential Back-off with Jitter

The delay calculation implements industry-standard exponential back-off with capped growth and randomization:

delay = min(maxDelayMs, baseDelayMs * 2^(attempt - 1))
jitter = delay * random() * 0.2  // ≤ 20% of calculated delay
finalDelay = delay + jitter

Jitter prevents thundering herd scenarios when multiple generation requests fail simultaneously.

withGenerationRetry: The Main Execution Loop

The withGenerationRetry(operation, options) function orchestrates the entire retry lifecycle. Its control flow handles four distinct paths:

1. Success with acceptable result – Return immediately if shouldRetryResult is absent or returns false
2. Success requiring retry – Emit onRetry with reason "empty result" and back-off
3. Abort error – Re-throw immediately, bypassing retry logic
4. Retryable error – Emit onRetry with derived reason and back-off; re-throw if max attempts exhausted

// Simplified structure from generation-retry.ts
const result = await withGenerationRetry(
  async (attempt) => await aiCall(systemPrompt, userPrompt, visionImages),
  {
    label: `slide-${outline.title}`,
    maxRetries: 5,
    baseDelayMs: 1000,
    onRetry: (e) => log.info(`Retry ${e.attempt}/${e.maxAttempts}: ${e.reason}`),
  },
);

The attempt parameter passed to the operation callback enables sophisticated logging or alternative routing on subsequent tries.

isAbortError: Cross-Environment Abort Detection

OpenMAIC normalizes abort detection across:

• Browsers – DOMException with name === 'AbortError'
• Node.js – Native AbortError instances
• Test environments – Plain objects with name property

This ensures AbortController signals propagate correctly regardless of execution context.

onRetry Callback: Observable Progress

The GenerationRetryEvent payload includes:

• label – Operation identifier
• attempt / maxAttempts – Progress context
• nextDelayMs – When the next attempt will occur
• reason – Human-readable classification ("rate limit", "empty result", etc.)

This enables UI components to display "Retrying in 3s..." messages and analytics pipelines to track retry patterns.

Integration Points in the Codebase

Scene Generation Pipeline

In packages/@openmaic/generation/src/scene-generator.ts, three core functions employ the retry wrapper:

• generateSlideContent() – Wraps slide LLM calls
• generateQuizContent() – Wraps quiz generation
• generateWidgetContent() – Wraps interactive component creation

Each passes a contextual label containing the outline title, enabling precise failure attribution in distributed traces.

Classroom Generation Server

The server-side entry point at lib/server/classroom-generation.ts applies the same pattern to higher-level planner orchestration. This demonstrates the utility's flexibility: it handles not just raw LLM calls but composite operations spanning multiple model invocations.

Practical Code Examples

Basic LLM Call Wrapping

import { withGenerationRetry } from '@openmaic/generation';
import { aiCall } from './ai-service';

async function generateSlide(outline) {
  const prompts = buildPrompt(outline);
  const result = await withGenerationRetry(
    async (attempt) => await aiCall(prompts.system, prompts.user),
    {
      label: `slide-${outline.title}`,
      maxRetries: 4,
      baseDelayMs: 1500,
      onRetry: (e) => console.log(`Retry ${e.attempt}: ${e.reason} → ${e.nextDelayMs}ms`),
    },
  );
  return parseJsonResponse(result);
}

Result-Based Retry: Empty Array Handling

await withGenerationRetry(
  async () => await fetchActions(),
  {
    label: 'actions-fetch',
    maxRetries: 3,
    shouldRetryResult: (arr) => Array.isArray(arr) && arr.length === 0,
    onRetry: (e) => console.warn(`Empty result, retrying: ${e.reason}`),
  },
);

Test-Injectable Sleep for Deterministic Timing

await withGenerationRetry(
  async () => await aiCall(prompts),
  {
    label: 'test-retry',
    sleep: (ms) => new Promise((res) => setTimeout(res, ms)),
    // In Jest: replace with jest.advanceTimersByTime(ms) for instant resolution
  },
);

AbortController Integration

const controller = new AbortController();
setTimeout(() => controller.abort(), 5000);

try {
  await withGenerationRetry(
    async () => await aiCall(prompts),
    { label: 'abort-demo', signal: controller.signal },
  );
} catch (e) {
  if (e.name === 'AbortError') {
    console.log('Generation aborted by user');
  }
}

Key Source Files

File	Role	Link
packages/@openmaic/generation/src/generation-retry.ts	Core implementation: helper, error classification, back-off logic	View source
packages/@openmaic/generation/src/scene-generator.ts	Scene generation integration (slides, quizzes, widgets)	View source
lib/server/classroom-generation.ts	Classroom planner pipeline integration	View source
tests/server/generation-retry.test.ts	Unit tests: retry triggers, back-off math, error classification	View source

Summary

• Centralization – OpenMAIC implements generation retry logic in a single file (generation-retry.ts) consumed across all generation workflows
• Intelligent classification – isRetryableGenerationError recursively inspects errors for HTTP codes, network messages, and timeout indicators
• Configurable back-off – Exponential delays with 20% jitter prevent thundering herds while bounding maximum wait times
• Dual trigger paths – Retries fire on both exceptions (isRetryableGenerationError) and successful but unsatisfactory results (shouldRetryResult)
• Full observability – The onRetry callback exposes structured events for logging, analytics, and UI progress indicators
• Testability – Injectable sleep and random functions enable deterministic unit tests without real delays

Frequently Asked Questions

How does OpenMAIC determine whether an error should trigger a retry?

The isRetryableGenerationError function in generation-retry.ts applies hierarchical checks: explicit isRetryable flags, specific HTTP status codes (408, 409, 425, 429), network-related message substrings (rate limit, ECONNRESET), timeout indicators, and recursive traversal of nested error fields (cause, lastError, errors). Errors classified as client errors (400-404, 422) bypass retry logic entirely.

Can OpenMAIC retry generation based on the result value, not just exceptions?

Yes. The shouldRetryResult predicate in GenerationRetryOptions enables result-driven retries. Callers provide a function that inspects the successful return value and returns true to trigger another attempt. This pattern handles cases where LLMs return valid but semantically empty responses, such as empty arrays or placeholder strings.

How does the retry back-off calculation work in OpenMAIC?

The retryDelayMs function implements capped exponential back-off with jitter. For attempt n, the base delay equals baseDelayMs * 2^(n-1), capped at maxDelayMs. A random jitter up to 20% of this value is added. The default baseDelayMs of 1000ms yields retry delays of approximately 1s, 2s, 4s, 8s, etc., with variation preventing synchronized retry storms.

Where is the generation retry logic tested in OpenMAIC?

Comprehensive unit tests reside in tests/server/generation-retry.test.ts. These verify: retry activation on null results, suppression for non-retryable HTTP codes, mathematical correctness of exponential back-off, proper handling of nested error structures, and immediate propagation of abort signals. The tests leverage the injectable sleep parameter to execute synchronously with Jest fake timers.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}