# How Model Selection Works When Multiple AI Providers Are Supported in screenshot-to-code

> Discover how screenshot-to-code dynamically selects AI models from OpenAI, Anthropic, and Gemini. Learn to optimize LLM combinations for your needs.

- Repository: [Abi Raja/screenshot-to-code](https://github.com/abi/screenshot-to-code)
- Tags: internals
- Published: 2026-03-02

---

**The screenshot-to-code repository implements a deterministic, rule-based model selection stage that evaluates available API keys (OpenAI, Anthropic, and Gemini) alongside request parameters to dynamically choose the optimal LLM combinations from predefined constant tuples.**

The abi/screenshot-to-code project orchestrates code generation across diverse AI providers through an intelligent routing mechanism. When processing requests, the backend must decide whether to invoke OpenAI's GPT models, Anthropic's Claude, or Google's Gemini—often mixing providers to maximize output quality. This article examines the `ModelSelectionStage` class in [`backend/routes/generate_code.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/routes/generate_code.py) to reveal exactly how the system prioritizes providers and balances workload across multiple LLMs.

## The ModelSelectionStage Entry Point

When a generation request arrives, the backend instantiates a **ModelSelectionStage** object and calls its asynchronous `select_models` method. This method serves as the primary entry point for all model selection decisions, receiving the **generation type** (`"create"` or `"update"`), the **input mode** (`"text"`, `"image"`, or `"video"`), and the three possible API keys (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`).

The method determines the number of variants to generate using `NUM_VARIANTS` (default **4**) for standard requests or `NUM_VARIANTS_VIDEO` (default **2**) for video mode, then delegates the actual provider selection to a private helper named `_get_variant_models`. According to the source code in [`backend/routes/generate_code.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/routes/generate_code.py) (lines 53-71), this architecture cleanly separates request validation from provider-specific logic.

## Provider Prioritization Logic in `_get_variant_models`

The `_get_variant_models` helper contains the complete decision matrix for selecting models based on available API keys. Located in [`backend/routes/generate_code.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/routes/generate_code.py), this method evaluates conditions in priority order to determine which predefined model tuple to return from [`backend/routes/model_choice_sets.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/routes/model_choice_sets.py).

### Video Mode Requirements

If `input_mode == "video"`, the system strictly requires a Gemini API key and returns `VIDEO_VARIANT_MODELS`. This mode exclusively uses Google's Gemini models to process video inputs, raising an error if the Gemini key is missing.

### Edit/Update Mode Optimization

When the generation type is an edit or update and both Gemini and OpenAI keys are present, the system prefers one Gemini and one fast OpenAI model. Specifically, it selects `GEMINI_3_FLASH_PREVIEW_MINIMAL` paired with `GPT_5_2_CODEX_LOW` to balance quality with responsiveness for iterative editing tasks.

### Multi-Provider Scenarios

If **all three API keys** are present, the method selects from `ALL_KEYS_MODELS_*` constants depending on context. For text creation tasks, it uses `ALL_KEYS_MODELS_TEXT_CREATE` (typically including Gemini Flash minimal, GPT‑5.2 Codex high, Claude Opus, and Gemini 3 Pro preview low). For updates, it uses `ALL_KEYS_MODELS_UPDATE`, falling back to `ALL_KEYS_MODELS_DEFAULT` for other cases.

### Fallback Strategies

When only two keys are available, the system uses paired constants: `GEMINI_ANTHROPIC_MODELS`, `GEMINI_OPENAI_MODELS`, or `OPENAI_ANTHROPIC_MODELS`. If only a single provider is configured, it falls back to provider-specific tuples like `OPENAI_ONLY_MODELS` (containing high and medium Codex variants), `ANTHROPIC_ONLY_MODELS`, or `GEMINI_ONLY_MODELS`. When no keys are detected, the stage raises an exception prompting the user to add an API key.

## Round-Robin Variant Distribution

After selecting the appropriate model tuple, the system must ensure the number of variants matches the requested count. The implementation cycles through the chosen model list using modulo arithmetic:

```python
selected_models = []
for i in range(num_variants):
    selected_models.append(models[i % len(models)])

```

This round-robin approach, found in [`backend/routes/generate_code.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/routes/generate_code.py) (lines 44-48), guarantees that if the requested variant count exceeds the available models, the system distributes work evenly across providers. For example, with two models but four variants requested, each model handles two generation tasks.

## Configuration Constants and Defaults

The variant counts are controlled by global constants defined in [`backend/config.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/config.py). **NUM_VARIANTS** defaults to **4** for standard image and text generations, while **NUM_VARIANTS_VIDEO** defaults to **2** for video processing tasks. These values drive how many times the model selection logic cycles through the available provider tuples, directly impacting the number of alternative code generations the user receives.

## Error Handling for Missing API Keys

The selection stage validates API key presence before attempting model assignment. If required keys are missing—for instance, attempting video generation without a Gemini key—the `_get_variant_models` method raises an exception that bubbles up to the HTTP handler in [`backend/routes/generate_code.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/routes/generate_code.py) (lines 84-92). The handler surfaces clear error messages to the client, preventing cryptic failures later in the generation pipeline.

## Practical Implementation Examples

### Using the Selector with All Providers

This example demonstrates model selection when all three API keys are configured for a text creation task:

```python
from backend.routes.generate_code import ModelSelectionStage
from backend.routes.model_choice_sets import (
    VIDEO_VARIANT_MODELS, ALL_KEYS_MODELS_DEFAULT,
)
import asyncio

async def demo():
    # Simulate an environment where all three keys are present

    selector = ModelSelectionStage(throw_error=lambda msg: asyncio.sleep(0))
    models = await selector.select_models(
        generation_type="create",
        input_mode="text",            # could be "image" or "video"

        openai_api_key="sk-...",      # any non‑None value

        anthropic_api_key="sk-ant...",# any non‑None value

        gemini_api_key="sk-gem...",   # any non‑None value

    )
    print([m.value for m in models])

asyncio.run(demo())

```

When executed with `generation_type="create"` and `input_mode="text"`, this returns the `ALL_KEYS_MODELS_TEXT_CREATE` tuple containing four distinct models across the three providers.

### Video Mode with Gemini Only

Video processing requires Gemini access and ignores other providers:

```python
async def video_demo():
    selector = ModelSelectionStage(throw_error=lambda msg: asyncio.sleep(0))
    models = await selector.select_models(
        generation_type="create",
        input_mode="video",
        openai_api_key=None,
        anthropic_api_key=None,
        gemini_api_key="sk-gemini",   # only Gemini key needed

    )
    # Will be VIDEO_VARIANT_MODELS (two Gemini models)

    print([m.value for m in models])

asyncio.run(video_demo())

```

### OpenAI-Only Fallback

When only OpenAI is configured, the system automatically restricts itself to OpenAI models:

```python
async def openai_only_demo():
    selector = ModelSelectionStage(throw_error=lambda msg: asyncio.sleep(0))
    models = await selector.select_models(
        generation_type="update",
        input_mode="image",
        openai_api_key="sk-openai",
        anthropic_api_key=None,
        gemini_api_key=None,
    )
    # Returns the OPENAI_ONLY_MODELS tuple (high & medium Codex)

    print([m.value for m in models])

asyncio.run(openai_only_demo())

```

## Summary

- **Model selection** occurs through `ModelSelectionStage.select_models` in [`backend/routes/generate_code.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/routes/generate_code.py), which evaluates request context and available API keys.
- **Decision logic** resides in `_get_variant_models`, using a priority matrix to select from constant tuples defined in [`backend/routes/model_choice_sets.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/routes/model_choice_sets.py).
- **Video mode** strictly requires Gemini API keys and uses `VIDEO_VARIANT_MODELS`.
- **Variant distribution** employs round-robin cycling to match `NUM_VARIANTS` (4) or `NUM_VARIANTS_VIDEO` (2) from [`backend/config.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/config.py).
- **Error handling** validates key presence before processing, surfacing clear messages for missing required providers.

## Frequently Asked Questions

### What happens if I only provide one AI provider's API key?

The system falls back to provider-specific model tuples defined in [`backend/routes/model_choice_sets.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/routes/model_choice_sets.py). For example, with only an OpenAI key, it selects `OPENAI_ONLY_MODELS` containing high and medium Codex variants, ensuring the application functions with single-provider configurations.

### Why does video input mode require a Gemini API key?

Video processing relies on Gemini's multimodal capabilities for analyzing video content. The `_get_variant_models` function explicitly checks for `input_mode == "video"` and returns `VIDEO_VARIANT_MODELS` only when a Gemini key is present, as other providers currently lack the necessary video understanding capabilities for this workflow.

### How does the system determine how many model variants to generate?

The variant count comes from `NUM_VARIANTS` (default 4) for standard generations or `NUM_VARIANTS_VIDEO` (default 2) for video mode, both defined in [`backend/config.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/config.py). These constants control how many times the selected models cycle to produce alternative code generations for a single request.

### Can I customize which specific models are used for each provider combination?

Yes, by modifying the constant tuples in [`backend/routes/model_choice_sets.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/routes/model_choice_sets.py) (lines 1-68). Each tuple—such as `ALL_KEYS_MODELS_TEXT_CREATE` or `GEMINI_ANTHROPIC_MODELS`—contains specific `Llm` enum values from [`backend/llm.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/llm.py) that you can adjust to use different model variants like GPT-4, Claude Opus, or Gemini Pro.