How Selectors Adapt Input Schemas Between Different Providers in OpenMontage

Selectors in OpenMontage serve as intelligent routing layers that normalize canonical input schemas into provider-specific contracts, enabling seamless interoperability between heterogeneous video and image generation backends.

OpenMontage employs selector tools such as VideoSelector and ImageSelector to shield AI agents from the complexity of provider-specific implementations. This architecture allows agents to use a stable, high-level API while the selectors handle translation, capability filtering, and routing across disparate providers like FAL, Wan, and ComfyUI. Understanding how selectors adapt input schemas between different providers in OpenMontage reveals the engine's flexibility in managing multi-backend workflows.

Selector Architecture as a Routing Layer

Selectors function as thin abstraction layers that inherit from BaseTool defined in tools/base_tool.py. Rather than exposing agents directly to provider heterogeneity, selectors expose a unified contract while internally translating requests to match each backend's expected parameters.

The core selectors include:

• VideoSelector (tools/video/video_selector.py): Handles video generation operations including text-to-video and image-to-video.
• ImageSelector (tools/graphics/image_selector.py): Manages image generation and manipulation tasks.

Both tools auto-discover available providers through the central ToolRegistry and dynamically build fallback lists based on declared capabilities.

Input Schema Normalization

Selectors receive a canonical schema through their input_schema definition. Before delegation to concrete providers, they perform key translations to align with provider-specific contracts.

Key Mapping and Field Translation

The execute method in each selector handles critical field transformations:

• prompt → query: Required for stock-search tools that expect search queries rather than generation prompts.
• model_name → model: Adapts to providers using the shorter model field.
• n → num_images: Translates quantity parameters for providers using different count naming conventions.

In VideoSelector, lines 34-40 perform this mapping before delegation, while ImageSelector implements analogous logic in lines 44-52.

Image Reference Consolidation

Selectors normalize various image input formats into provider-compatible structures:

• ImageSelector (lines 52-60): Consolidates image_path, image_url, and related fields into a unified images array.
• VideoSelector (lines 42-50): Handles reference_image_path inputs by uploading local assets and converting them to image_url parameters expected by downstream providers.

This ensures providers receive properly formatted references regardless of how the agent initially specified them.

Provider Discovery and Capability Filtering

Selectors dynamically discover and filter providers using capability metadata rather than static configurations.

Auto-Discovery via ToolRegistry

The discovery process begins in tools/tool_registry.py (lines 270-276), where the registry identifies all tools implementing specific capabilities like video_generation or image_generation. Selectors query this registry to build fallback_tools lists, exposing available backends to downstream contracts without hardcoded dependencies.

Operation-Based Provider Filtering

Selectors enforce capability gating through the _filter_candidates method, which inspects each provider's supports dictionary and input_schema:

• VideoSelector (lines 64-78): Excludes image_selector from fallbacks for motion-required operations like image_to_video, ensuring only video-capable providers are considered.
• Custom Parameters: Filters tools based on required parameters such as image_url, reference_image_urls, or custom_workflow support.

This prevents delegation to providers lacking necessary features for the requested operation.

Custom Workflow Routing

When agents supply ComfyUI workflow definitions via workflow_json or workflow_path parameters, selectors route requests exclusively to compatible backends. The _custom_workflow_eligible method (lines 45-60 in both selectors) checks for custom_workflow capability advertisements, bypassing standard model-based providers when custom node graphs are specified.

Scored Provider Selection

After normalization and filtering, selectors determine the optimal backend using rank_providers from lib/scoring.py. This returns ProviderScore objects that evaluate providers across multiple dimensions.

The _select_best_tool method (VideoSelector, lines 81-33) selects the highest-scoring provider while respecting preferred_provider hints if the specified provider scores within the preferred_provider_gap tolerance threshold. This balances explicit user preferences against quality scores.

Result Enrichment and Audit Trails

Once execution completes, selectors inject comprehensive metadata into results:

• selected_tool: The specific tool instance used.
• selected_provider: The backend provider name.
• selection_reason: Rationale for the routing decision.
• provider_score: Quantified suitability score.
• alternatives_considered: List of evaluated but rejected providers.

This enrichment creates transparent audit trails, allowing agents to understand routing decisions without requiring knowledge of individual provider contracts.

Code Examples

# Example 1: Text-to-video generation (stock fallback allowed)

from tools.video.video_selector import VideoSelector

selector = VideoSelector()
result = selector.execute({
    "prompt": "A futuristic city skyline at sunset",
    "operation": "text_to_video",          # stock providers will see 'query'

    "preferred_provider": "auto"
})

print(result.data["selected_provider"])   # e.g. "fal" or "ark"

print(result.data["selection_reason"])

# Example 2: Image-to-video with a local reference image

from tools.video.video_selector import VideoSelector

selector = VideoSelector()
result = selector.execute({
    "prompt": "A dragon flying over mountains",
    "operation": "image_to_video",
    "reference_image_path": "assets/dragon.jpg",   # will be uploaded → image_url

    "preferred_provider": "wan"
})

# The selector uploads the image (via tools.video._shared.upload_image_fal)

# and passes `image_url` to the chosen provider.

print(result.data["selected_tool"])        # e.g. "wan_video"

# Example 3: Image generation with a custom ComfyUI workflow

from tools.graphics.image_selector import ImageSelector

selector = ImageSelector()
result = selector.execute({
    "prompt": "A cyberpunk street scene",
    "operation": "generate",
    "workflow_json": "...",          # custom workflow definition

    "output_node": "OUTPUT_1",
    "preferred_provider": "auto"
})

# The selector routes only to providers that support `custom_workflow`.

print(result.data["selected_provider"])  # e.g. "comfyui_image"

Summary

• Selectors normalize input schemas by mapping canonical keys to provider-specific fields (e.g., prompt → query) within their execute methods.
• Provider discovery occurs dynamically through ToolRegistry (lines 270-276), eliminating hardcoded backend lists.
• Capability filtering via _filter_candidates and supports dictionaries ensures only compatible providers handle specific operations.
• Custom workflow routing uses _custom_workflow_eligible to isolate requests to ComfyUI-capable providers.
• Scored selection through rank_providers and _select_best_tool optimizes provider choice while respecting preference gaps.
• Result enrichment provides complete audit trails including selection reasoning and alternative considerations.

Frequently Asked Questions

What is the primary function of selectors in OpenMontage?

Selectors act as schema adapters and routing orchestrators that expose a stable, high-level API to agents while handling provider-specific translations internally. They manage the complexity of heterogeneous backend contracts, capability filtering, and optimal provider selection without requiring agents to understand individual provider implementations.

How does VideoSelector handle reference images for image-to-video operations?

VideoSelector accepts local paths via reference_image_path and manages the upload process through tools.video._shared.upload_image_fal, converting local assets to image_url parameters expected by providers (lines 42-50). This shields agents from managing upload logistics and URL generation.

Can selectors route requests to specific providers based on custom workflow requirements?

Yes. When callers provide workflow_json or workflow_path parameters, the _custom_workflow_eligible method (lines 45-60) filters the provider pool to include only those advertising custom_workflow support, ensuring ComfyUI workflows route exclusively to compatible backends regardless of standard model availability.

How does OpenMontage filter incompatible providers during execution?

Selectors use the _filter_candidates method to inspect provider supports dictionaries and input_schema definitions. For example, VideoSelector excludes image_selector from fallbacks for motion operations (lines 64-78), and all selectors verify that providers declare required parameters like image_url or custom_workflow before considering them for delegation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}