How Screenshot-to-Code Generates and Manages Multiple Code Variants per Request

The variant system creates several independent code completions for each generation request by running parallel LLM agents and synchronizing the results through WebSocket messages to a responsive frontend grid.

The abi/screenshot-to-code repository implements a configurable pipeline that produces multiple interpretations of a screenshot within a single request. By orchestrating distinct model selections and parallel execution strategies, the backend delivers diverse code variants while the frontend maintains real-time synchronization across an adaptive user interface.

Configuration and Scaling

The entire pipeline scales through a single configuration value. In backend/config.py, the NUM_VARIANTS constant defines how many independent generations run per request, defaulting to 4.

# backend/config.py

NUM_VARIANTS = 4

Changing this value automatically adjusts both the backend's parallel execution count and the frontend's grid capacity without requiring additional code modifications.

Backend Pipeline Architecture

A composed Pipeline in backend/routes/generate_code.py orchestrates the request through specialized middleware stages. Two critical stages manage the variant lifecycle:

Model Selection Strategy

The ModelSelectionStage distributes available LLMs across the requested number of variants. It cycles through models (Claude, OpenAI, Anthropic, Gemini) and repeats the sequence when variants exceed model availability, storing selections in context.variant_models.

# backend/routes/generate_code.py

def _get_variant_models(..., num_variants, ...) -> List[Llm]:
    # Example with two models available → repeats A, B, A, B, A for 5 variants

    base = [Llm.CLAUDE_3_7_SONNET, Llm.GPT_4_1_NANO_2025_04_14]
    return [base[i % len(base)] for i in range(num_variants)]

Parallel Generation with AsyncIO

The AgenticGenerationStage executes each variant through its own Agent instance using asyncio.gather for concurrent processing. Each task runs independently through the tool-calling chain defined in backend/agent/runner.py.

# backend/routes/generate_code.py

tasks.append(asyncio.create_task(self._run_variant(index, model, prompt_messages)))

# Results are stored as {index: completion}

variant_completions[index] = result

Successful completions populate context.variant_completions as a dictionary mapping variant indices to generated code strings. The WebSocketCommunicator then streams each result to the client using the "setCode" message type.

Frontend State Synchronization

The WebSocket client receives a structured message sequence beginning with "variantCount", which prepares the UI for the incoming data stream. This message updates the state store defined in frontend/src/store/project-store.ts.

// frontend/src/generateCode.ts
socket.onmessage = (event) => {
  const msg = JSON.parse(event.data);
  if (msg.type === "variantCount") {
    store.setVariantCount(msg.value);  // updates UI grid
  }
};

Responsive Grid Layout

The Variants.tsx component dynamically calculates column distribution based on the variant count, creating layouts ranging from 2-column grids to 4-column arrangements.

// frontend/src/components/variants/Variants.tsx
const columns = variantCount < 4 ? 2 : variantCount < 7 ? 3 : 4;

Keyboard shortcuts (⌥1, ⌥2, etc.) provide global navigation between panes, allowing users to compare implementations side-by-side before selecting their preferred variant.

Summary

• Configuration: NUM_VARIANTS in backend/config.py controls pipeline scale
• Model Distribution: The backend cycles through available LLMs using modulo arithmetic in _get_variant_models
• Parallel Execution: asyncio.gather runs independent Agent instances simultaneously for each variant
• State Management: Results are collected in context.variant_completions and streamed via WebSocket
• UI Adaptation: The frontend receives variantCount first, then renders a responsive grid in Variants.tsx with keyboard navigation support

Frequently Asked Questions

How does the system handle more variants than available LLM models?

The ModelSelectionStage uses modulo arithmetic to cycle through the base model list. If you request 5 variants but only have 2 models configured, the assignment pattern becomes A, B, A, B, A, ensuring every variant receives a model assignment without duplication errors.

What WebSocket message initiates the variant grid rendering?

The frontend receives a "variantCount" message before any code completions arrive. This signals frontend/src/store/project-store.ts to initialize the grid layout in Variants.tsx, preparing the UI to receive the subsequent "setCode" messages containing the actual generated code.

Are code variants generated sequentially or in parallel?

Variants execute in parallel using asyncio.gather within the AgenticGenerationStage. This approach minimizes total generation latency by running multiple LLM requests simultaneously rather than waiting for each completion serially.

Where is the default number of variants configured?

The default quantity is defined as NUM_VARIANTS in backend/config.py (set to 4). This constant drives the entire pipeline, from the number of parallel tasks created in handle_generate to the grid columns calculated in the frontend.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}