# How OpenMAIC Implements Generation Retry Logic: A Code-Deep Analysis

> Explore OpenMAIC's generation retry logic implementation. Discover how a centralized helper uses exponential back-off and intelligent error handling for robust async operations.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: deep-dive
- Published: 2026-09-03

---

**OpenMAIC implements generation retry logic through a centralized `withGenerationRetry` helper in `packages/@openmaic/generation/src/generation-retry.ts`, which wraps async operations with configurable exponential back-off, intelligent error classification, and result-based retry predicates.**

The THU-MAIC/OpenMAIC repository provides a sophisticated, declarative approach to handling transient failures in LLM-powered generation workflows. Rather than scattering retry logic across dozens of call sites, the codebase centralizes resilience patterns in a single, well-typed utility that every generation flow—from slides to quizzes to interactive widgets—can leverage consistently.

## Core Architecture of the Retry System

The generation retry logic in OpenMAIC rests on six integrated components, all implemented in the same source file.

### GenerationRetryOptions<T>: Configuration Interface

The `GenerationRetryOptions<T>` interface exposes fine-grained control over retry behavior. Parameters include:

- **`label`** – Human-readable identifier for logging
- **`maxRetries`** – Maximum retry attempts (default varies by caller)
- **`baseDelayMs`** / **`maxDelayMs`** – Back-off timing bounds
- **`signal`** – AbortController integration
- **`sleep`** – Injectable sleep function for testability
- **`random`** – Jitter source (default: `Math.random`)
- **`shouldRetryResult`** – Predicate to trigger retry based on returned value
- **`onRetry`** – Callback for progress surfacing (UI notifications, logs)

### isRetryableGenerationError: Error Classification Engine

This function performs deep inspection of errors to determine retry eligibility. As implemented in [`generation-retry.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/generation-retry.ts), it evaluates:

| Check | Implementation |
|-------|---------------|
| Explicit flag | `error.isRetryable === true` |
| HTTP status codes | Retryable: `408, 409, 425, 429`; Non-retryable: `400-404, 422` |
| Network indicators | Substring matches: `rate limit`, `ECONNRESET`, `ETIMEDOUT`, etc. |
| Timeout detection | `error.name === 'TimeoutError'` or similar |
| Nested error traversal | Recursively inspects `cause`, `lastError`, `errors` fields |

This recursive design handles wrapped errors from HTTP clients, LLM SDKs, and intermediate processors without flattening useful context.

### retryDelayMs: Exponential Back-off with Jitter

The delay calculation implements industry-standard exponential back-off with capped growth and randomization:

```typescript
delay = min(maxDelayMs, baseDelayMs * 2^(attempt - 1))
jitter = delay * random() * 0.2  // ≤ 20% of calculated delay
finalDelay = delay + jitter

```

Jitter prevents thundering herd scenarios when multiple generation requests fail simultaneously.

### withGenerationRetry: The Main Execution Loop

The `withGenerationRetry(operation, options)` function orchestrates the entire retry lifecycle. Its control flow handles four distinct paths:

1. **Success with acceptable result** – Return immediately if `shouldRetryResult` is absent or returns `false`
2. **Success requiring retry** – Emit `onRetry` with reason "empty result" and back-off
3. **Abort error** – Re-throw immediately, bypassing retry logic
4. **Retryable error** – Emit `onRetry` with derived reason and back-off; re-throw if max attempts exhausted

```typescript
// Simplified structure from generation-retry.ts
const result = await withGenerationRetry(
  async (attempt) => await aiCall(systemPrompt, userPrompt, visionImages),
  {
    label: `slide-${outline.title}`,
    maxRetries: 5,
    baseDelayMs: 1000,
    onRetry: (e) => log.info(`Retry ${e.attempt}/${e.maxAttempts}: ${e.reason}`),
  },
);

```

The `attempt` parameter passed to the operation callback enables sophisticated logging or alternative routing on subsequent tries.

### isAbortError: Cross-Environment Abort Detection

OpenMAIC normalizes abort detection across:

- **Browsers** – `DOMException` with `name === 'AbortError'`
- **Node.js** – Native `AbortError` instances
- **Test environments** – Plain objects with `name` property

This ensures `AbortController` signals propagate correctly regardless of execution context.

### onRetry Callback: Observable Progress

The `GenerationRetryEvent` payload includes:

- `label` – Operation identifier
- `attempt` / `maxAttempts` – Progress context
- `nextDelayMs` – When the next attempt will occur
- `reason` – Human-readable classification ("rate limit", "empty result", etc.)

This enables UI components to display "Retrying in 3s..." messages and analytics pipelines to track retry patterns.

## Integration Points in the Codebase

### Scene Generation Pipeline

In `packages/@openmaic/generation/src/scene-generator.ts`, three core functions employ the retry wrapper:

- `generateSlideContent()` – Wraps slide LLM calls
- `generateQuizContent()` – Wraps quiz generation
- `generateWidgetContent()` – Wraps interactive component creation

Each passes a contextual label containing the outline title, enabling precise failure attribution in distributed traces.

### Classroom Generation Server

The server-side entry point at [`lib/server/classroom-generation.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/classroom-generation.ts) applies the same pattern to higher-level planner orchestration. This demonstrates the utility's flexibility: it handles not just raw LLM calls but composite operations spanning multiple model invocations.

## Practical Code Examples

### Basic LLM Call Wrapping

```typescript
import { withGenerationRetry } from '@openmaic/generation';
import { aiCall } from './ai-service';

async function generateSlide(outline) {
  const prompts = buildPrompt(outline);
  const result = await withGenerationRetry(
    async (attempt) => await aiCall(prompts.system, prompts.user),
    {
      label: `slide-${outline.title}`,
      maxRetries: 4,
      baseDelayMs: 1500,
      onRetry: (e) => console.log(`Retry ${e.attempt}: ${e.reason} → ${e.nextDelayMs}ms`),
    },
  );
  return parseJsonResponse(result);
}

```

### Result-Based Retry: Empty Array Handling

```typescript
await withGenerationRetry(
  async () => await fetchActions(),
  {
    label: 'actions-fetch',
    maxRetries: 3,
    shouldRetryResult: (arr) => Array.isArray(arr) && arr.length === 0,
    onRetry: (e) => console.warn(`Empty result, retrying: ${e.reason}`),
  },
);

```

### Test-Injectable Sleep for Deterministic Timing

```typescript
await withGenerationRetry(
  async () => await aiCall(prompts),
  {
    label: 'test-retry',
    sleep: (ms) => new Promise((res) => setTimeout(res, ms)),
    // In Jest: replace with jest.advanceTimersByTime(ms) for instant resolution
  },
);

```

### AbortController Integration

```typescript
const controller = new AbortController();
setTimeout(() => controller.abort(), 5000);

try {
  await withGenerationRetry(
    async () => await aiCall(prompts),
    { label: 'abort-demo', signal: controller.signal },
  );
} catch (e) {
  if (e.name === 'AbortError') {
    console.log('Generation aborted by user');
  }
}

```

## Key Source Files

| File | Role | Link |
|------|------|------|
| `packages/@openmaic/generation/src/generation-retry.ts` | Core implementation: helper, error classification, back-off logic | [View source](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/generation/src/generation-retry.ts) |
| `packages/@openmaic/generation/src/scene-generator.ts` | Scene generation integration (slides, quizzes, widgets) | [View source](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/generation/src/scene-generator.ts) |
| [`lib/server/classroom-generation.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/classroom-generation.ts) | Classroom planner pipeline integration | [View source](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/classroom-generation.ts) |
| [`tests/server/generation-retry.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/server/generation-retry.test.ts) | Unit tests: retry triggers, back-off math, error classification | [View source](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/server/generation-retry.test.ts) |

## Summary

- **Centralization** – OpenMAIC implements generation retry logic in a single file ([`generation-retry.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/generation-retry.ts)) consumed across all generation workflows
- **Intelligent classification** – `isRetryableGenerationError` recursively inspects errors for HTTP codes, network messages, and timeout indicators
- **Configurable back-off** – Exponential delays with 20% jitter prevent thundering herds while bounding maximum wait times
- **Dual trigger paths** – Retries fire on both exceptions (`isRetryableGenerationError`) and successful but unsatisfactory results (`shouldRetryResult`)
- **Full observability** – The `onRetry` callback exposes structured events for logging, analytics, and UI progress indicators
- **Testability** – Injectable `sleep` and `random` functions enable deterministic unit tests without real delays

## Frequently Asked Questions

### How does OpenMAIC determine whether an error should trigger a retry?

The `isRetryableGenerationError` function in [`generation-retry.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/generation-retry.ts) applies hierarchical checks: explicit `isRetryable` flags, specific HTTP status codes (`408, 409, 425, 429`), network-related message substrings (`rate limit`, `ECONNRESET`), timeout indicators, and recursive traversal of nested error fields (`cause`, `lastError`, `errors`). Errors classified as client errors (`400-404, 422`) bypass retry logic entirely.

### Can OpenMAIC retry generation based on the result value, not just exceptions?

Yes. The `shouldRetryResult` predicate in `GenerationRetryOptions` enables result-driven retries. Callers provide a function that inspects the successful return value and returns `true` to trigger another attempt. This pattern handles cases where LLMs return valid but semantically empty responses, such as empty arrays or placeholder strings.

### How does the retry back-off calculation work in OpenMAIC?

The `retryDelayMs` function implements capped exponential back-off with jitter. For attempt *n*, the base delay equals `baseDelayMs * 2^(n-1)`, capped at `maxDelayMs`. A random jitter up to 20% of this value is added. The default `baseDelayMs` of 1000ms yields retry delays of approximately 1s, 2s, 4s, 8s, etc., with variation preventing synchronized retry storms.

### Where is the generation retry logic tested in OpenMAIC?

Comprehensive unit tests reside in [`tests/server/generation-retry.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/server/generation-retry.test.ts). These verify: retry activation on `null` results, suppression for non-retryable HTTP codes, mathematical correctness of exponential back-off, proper handling of nested error structures, and immediate propagation of abort signals. The tests leverage the injectable `sleep` parameter to execute synchronously with Jest fake timers.