What Happens When Generation Variants Fail During Concurrent Execution in screenshot-to-code

When generation variants fail during concurrent execution in the screenshot-to-code application, the system isolates the error to the specific variant, marks its status as error, and allows all other variants to continue processing uninterrupted.

The screenshot-to-code repository by abi implements a fault-tolerant parallel generation system where multiple code variants are produced simultaneously from a single screenshot. Understanding how the codebase handles individual variant failures is crucial for building resilient AI-powered generation pipelines.

Architecture of Concurrent Variant Generation

The variant system is designed to be scalable and fault-tolerant, with each variant operating as an independent logical stream. The architecture spans both backend error detection and frontend state management.

Backend Error Detection

In backend/routes/generate_code.py, the WebSocketCommunicator.send_message method handles the dispatch of error states. When a variant-specific exception occurs—whether from an LLM API failure, tool crash, or network timeout—the backend captures the error and serializes it into a variantError message:

{
  "type": "variantError",
  "value": "Model quota exceeded",
  "variantIndex": 2
}

The backend sends this message via await self.send_message("variantError", error_message, index, None, None), where index represents the zero-based variant number that failed.

Frontend State Management

The frontend receives the error through frontend/src/generateCode.ts, which routes WebSocket responses based on the type field. When response.type === "variantError" is detected, the system invokes the onVariantError callback with the variant index and error message.

In frontend/src/App.tsx, the onVariantError handler orchestrates the state update by:

1. Logging the error for debugging purposes
2. Calling updateVariantStatus(commit.hash, variantIndex, "error", error)
3. Finalizing any open agent-event streams (thinking, assistant, or tool events) belonging to that variant

How Failed Variants Are Isolated

The failure flow demonstrates strict isolation between variants. When generation variants fail during concurrent execution, the following sequence occurs:

1. Exception Capture: The backend detects the failure within the specific variant's coroutine without propagating the exception to other parallel coroutines.

2. Message Dispatch: The WebSocketCommunicator pushes the error payload to the client while keeping the WebSocket connection open for remaining variants.

3. State Persistence: In frontend/src/store/project-store.ts, the updateVariantStatus implementation selectively updates only the failed variant:

variants: commit.variants.map((variant, index) =>
  index === numVariant 
    ? { ...variant, status, completedAt: status === "generating" ? undefined : Date.now(),
        errorMessage: status === "error" ? errorMessage : undefined }
    : variant
)

4. UI Feedback: The Variants component in frontend/src/components/variants/Variants.tsx renders a red badge (bg-red-500) for the failed variant while maintaining green or gray indicators for completed or generating variants.

Why Other Variants Continue Running

The system ensures that one failed variant does not abort the entire generation batch. Several implementation details enforce this continuity:

• Independent State Storage: Each variant's state resides in its own array element within the global store. The updateVariantStatus logic never clears or aborts sibling variants when updating a single entry.

• Non-Propagating Coroutines: The backend pipeline launches separate coroutines per variant, ensuring that an unhandled exception in one coroutine cannot cascade to others.

• Selective UI Focus: The updateSelectedVariantIndex logic explicitly avoids cancelling other variants when switching focus between generation results.

Users can observe successful outputs from variant 1 while variant 2 displays an error badge, and variant 3 continues generating.

Implementation Example

To handle generation variant errors in a custom component, implement the onVariantError callback when invoking generateCode:

generateCode(wsRef, settings, {
  onVariantUpdate: (variantIndex, code) => {
    // Handle successful code updates
  },
  onVariantError: (variantIndex, error) => {
    console.error(`Variant ${variantIndex} failed: ${error}`);
    // The UI automatically reflects the error via the store → Variants component
  },
});

When the backend reports a quota error for variant index 2, the UI immediately reflects:

• Variant 1: Green completion badge with generated code
• Variant 2: Red error badge with tooltip "Model quota exceeded"
• Variant 3: Gray generating indicator with active streaming

Summary

• Isolated Failures: The variantError message type contains only the failing variant's index, preventing error propagation across the concurrent generation set.
• Persistent State: Failed variants store status: "error" and errorMessage in the project store at frontend/src/store/project-store.ts, preserving the failure context for user review.
• Continuous Processing: The WebSocket connection remains active for all non-failing variants, allowing the generation pipeline to complete partial successes.
• Visual Feedback: The Variants component provides immediate visual distinction through color-coded status badges (red for error, green for complete, gray for generating).

Frequently Asked Questions

How does the backend communicate variant failures to the frontend?

The backend sends a WebSocket message with type: "variantError" through the WebSocketCommunicator.send_message method in backend/routes/generate_code.py. This message includes the variantIndex and error value, which the frontend routes to the appropriate handler in frontend/src/generateCode.ts.

Can I retry a single failed variant without regenerating all variants?

Yes. Because the system stores per-variant status independently in frontend/src/store/project-store.ts, you can re-issue a generation request targeting only the failed variant index. The store's updateVariantStatus function updates only the specific array element, leaving successful variants untouched.

Where is the error state stored for failed generation variants?

Error states are persisted in the Zustand-based project store at frontend/src/store/project-store.ts. The updateVariantStatus function writes the error status, timestamps the completedAt field, and stores the raw errorMessage string within the specific variant's object in the variants array.

Does a failed variant affect the WebSocket connection for other variants?

No. The WebSocket connection remains open and fully functional for all other variants. The backend handles each variant in a separate coroutine, so a failure in one variant's processing loop does not trigger a connection closure or interrupt the message stream for concurrent variants.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}