How the Provider Factory Creates Sessions for OpenAI, Anthropic, and Gemini in screenshot-to-code

The create_provider_session factory function in abi/screenshot-to-code uses enum-based membership checks against provider-specific model sets to instantiate the correct client session, handling authentication and tool serialization uniquely for each AI provider.

The abi/screenshot-to-code repository abstracts LLM provider complexity through a centralized factory pattern. Located in backend/agent/providers/factory.py, the create_provider_session function dynamically routes requests to the appropriate provider implementation based on the Llm enum value, eliminating the need for caller-side provider logic while standardizing tool handling and streaming interfaces.

Model Enumeration and Provider Classification

The foundation of the factory pattern resides in backend/llm.py (lines 5-70), where the Llm enum defines all supported models such as gpt-4.1-2025-04-14, claude-sonnet-4-6, and gemini-3-flash-preview.

The module maintains a MODEL_PROVIDER dictionary that maps every enum member to its provider string ("openai", "anthropic", or "gemini"). From this mapping, the system generates convenience sets—OPENAI_MODELS, ANTHROPIC_MODELS, and GEMINI_MODELS—that the factory uses for fast membership checking. This design allows the factory to determine provider affiliation in constant time by checking if model in OPENAI_MODELS rather than parsing string values.

Factory Logic and Session Instantiation

The create_provider_session function in backend/agent/providers/factory.py (lines 16-65) serves as the central instantiation point. It accepts a model enum, prompt messages, image generation flags, and provider-specific API keys, then returns a concrete ProviderSession subclass.

def create_provider_session(
    model: Llm,
    prompt_messages: list[ChatCompletionMessageParam],
    should_generate_images: bool,
    openai_api_key: Optional[str],
    openai_base_url: Optional[str],
    anthropic_api_key: Optional[str],
    gemini_api_key: Optional[str],
) -> ProviderSession:
    canonical_tools = canonical_tool_definitions(
        image_generation_enabled=should_generate_images
    )
    # OpenAI

    if model in OPENAI_MODELS:
        if not openai_api_key: raise Exception("OpenAI API key is missing.")
        client = AsyncOpenAI(api_key=openai_api_key, base_url=openai_base_url)
        return OpenAIProviderSession(
            client=client,
            model=model,
            prompt_messages=prompt_messages,
            tools=serialize_openai_tools(canonical_tools),
        )
    # Anthropic

    if model in ANTHROPIC_MODELS:
        if not anthropic_api_key: raise Exception("Anthropic API key is missing.")
        client = AsyncAnthropic(api_key=anthropic_api_key)
        return AnthropicProviderSession(
            client=client,
            model=model,
            prompt_messages=prompt_messages,
            tools=serialize_anthropic_tools(canonical_tools),
        )
    # Gemini

    if model in GEMINI_MODELS:
        if not gemini_api_key: raise Exception("Gemini API key is missing.")
        client = genai.Client(api_key=gemini_api_key)
        return GeminiProviderSession(
            client=client,
            model=model,
            prompt_messages=prompt_messages,
            tools=serialize_gemini_tools(canonical_tools),
        )
    raise ValueError(f"Unsupported model: {model.value}")

The factory first generates canonical tool definitions via canonical_tool_definitions, then serializes them using provider-specific functions to match each SDK's expected format.

OpenAI Session Creation Path

When the requested model exists in OPENAI_MODELS, the factory validates that openai_api_key is present, then instantiates AsyncOpenAI from the OpenAI SDK. It constructs an OpenAIProviderSession (defined in backend/agent/providers/openai.py, lines 75-107) by passing the client, model enum, original prompt messages, and tools processed through serialize_openai_tools.

Anthropic Session Creation Path

For models in ANTHROPIC_MODELS, the factory verifies anthropic_api_key and creates AsyncAnthropic. It returns an AnthropicProviderSession (located in backend/agent/providers/anthropic.py, lines 27-44) with tools serialized via serialize_anthropic_tools. This session handles the translation between OpenAI-style message formats and Claude's native message structure.

Gemini Session Creation Path

When the model belongs to GEMINI_MODELS, the factory checks for gemini_api_key and instantiates genai.Client from Google's SDK. The resulting GeminiProviderSession (found in backend/agent/providers/gemini.py, lines 26-45) receives tools processed through serialize_gemini_tools and manages Gemini-specific Content objects for streaming responses.

Concrete Provider Session Implementations

Each provider session class implements the streaming and tool-execution logic specific to its underlying SDK:

• OpenAIProviderSession in backend/agent/providers/openai.py manages streaming via the responses.create API, handling tool calls and assembling ProviderTurn results from OpenAI's event stream.
• AnthropicProviderSession in backend/agent/providers/anthropic.py converts the internal prompt format to Claude's message structure, streams Claude's delta responses, and extracts tool calls into the standard turn format.
• GeminiProviderSession in backend/agent/providers/gemini.py builds Gemini Content objects from the prompt messages, streams the model's response chunks, and parses function calls and text completions into the unified ProviderTurn interface.

Practical Implementation Examples

The factory enables seamless provider switching by changing only the model enum and corresponding API key:

# OpenAI GPT-4.1 session

session = create_provider_session(
    model=Llm.GPT_4_1_2025_04_14,
    prompt_messages=messages,
    should_generate_images=True,
    openai_api_key=os.getenv("OPENAI_API_KEY"),
    openai_base_url=None,
    anthropic_api_key=None,
    gemini_api_key=None,
)

# Anthropic Claude Opus 4.6 session

session = create_provider_session(
    model=Llm.CLAUDE_OPUS_4_6,
    prompt_messages=messages,
    should_generate_images=False,
    openai_api_key=None,
    openai_base_url=None,
    anthropic_api_key=os.getenv("ANTHROPIC_API_KEY"),
    gemini_api_key=None,
)

# Gemini 3 Flash Preview session

session = create_provider_session(
    model=Llm.GEMINI_3_FLASH_PREVIEW_HIGH,
    prompt_messages=messages,
    should_generate_images=False,
    openai_api_key=None,
    openai_base_url=None,
    anthropic_api_key=None,
    gemini_api_key=os.getenv("GEMINI_API_KEY"),
)

Summary

• Enum-based routing: The factory uses membership checks against OPENAI_MODELS, ANTHROPIC_MODELS, and GEMINI_MODELS to determine the correct provider without string parsing.
• Credential isolation: Each provider path validates its specific API key (openai_api_key, anthropic_api_key, or gemini_api_key) before instantiating the corresponding SDK client.
• Tool serialization: Canonical tool definitions are transformed via provider-specific serializers (serialize_openai_tools, serialize_anthropic_tools, serialize_gemini_tools) to match each API's schema requirements.
• Unified interface: All factory outputs implement the ProviderSession interface, abstracting provider-specific streaming and response handling from the rest of the application.

Frequently Asked Questions

How does the factory determine which AI provider to instantiate?

The factory checks if the requested Llm enum value exists in the provider-specific sets defined in backend/llm.py. If the model is in OPENAI_MODELS, it creates an OpenAIProviderSession; if in ANTHROPIC_MODELS, an AnthropicProviderSession; if in GEMINI_MODELS, a GeminiProviderSession.

What happens if an API key is missing for the requested provider?

The factory raises a specific exception immediately before client instantiation. For example, when routing to an OpenAI model, if openai_api_key is None or empty, the factory raises "OpenAI API key is missing." before attempting to create the AsyncOpenAI client.

Are tool definitions handled consistently across different providers?

The factory first generates canonical tool definitions via canonical_tool_definitions, then serializes them using provider-specific functions to match each SDK's requirements. While the internal representation is consistent, the serialization step ensures OpenAI, Anthropic, and Gemini each receive tools in their native schema format.

Can the created sessions handle streaming responses?

Yes, each concrete implementation (OpenAIProviderSession, AnthropicProviderSession, GeminiProviderSession) implements provider-specific streaming logic. The factory ensures that regardless of which provider session is created, the calling code can consume streaming responses and tool calls through the uniform ProviderSession interface.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}