# How OpenMontage's 7-Dimension Provider Scoring Engine Ranks AI Video Tools

> Discover OpenMontage's 7-dimension provider scoring engine. Learn how it objectively ranks AI video tools across task fit, quality, cost, and more for optimal selection.

- Repository: [Calesthio/OpenMontage](https://github.com/calesthio/OpenMontage)
- Tags: deep-dive
- Published: 2026-08-30

---

**OpenMontage uses a deterministic, multi-dimensional scoring system defined in [`lib/scoring.py`](https://github.com/calesthio/OpenMontage/blob/main/lib/scoring.py) that evaluates AI providers across seven weighted dimensions—task fit, output quality, control, reliability, cost efficiency, latency, and continuity—to rank tools like `video_selector` and select the optimal provider for each generation task.**

The OpenMontage framework employs a sophisticated **7-dimension provider scoring engine** to intelligently route generation tasks to the most suitable AI provider. This deterministic system, implemented in [`lib/scoring.py`](https://github.com/calesthio/OpenMontage/blob/main/lib/scoring.py), evaluates candidate tools across multiple quality and operational metrics, ensuring optimal provider selection for capabilities like video generation.

## The Seven Dimensions of Provider Evaluation

The scoring engine evaluates every candidate provider across seven distinct dimensions, each returning a normalized float between 0 and 1. These dimensions capture both technical capability and operational fitness for the specific task context.

### Task Fit and Output Quality

**Task fit** measures how closely a provider’s advertised `best_for` attributes match the user’s intent and style requirements. The `_compute_task_fit` function calculates this using keyword overlap and synonym expansion against the task description.

**Output quality** represents the expected fidelity of the generated asset. This dimension derives its score from either historic `quality_score` data or the provider’s stability tier, where `production` tiers map to 0.9, `beta` to 0.7, and similar gradations.

### Control and Reliability

**Control** quantifies the amount of creative control the provider exposes through features like `controlnet`, `reference_image` support, and other fine-tuning parameters. The `_compute_control` function weights these feature flags to produce the final score.

**Reliability** indicates the probability that the tool will successfully complete the request. This score draws from historic success rates or current operational status, calculated within the `score_provider` function (lines 100-110).

### Cost, Latency, and Continuity

**Cost efficiency** evaluates value-for-money by comparing estimated execution costs against remaining budget constraints. The `_compute_cost_efficiency` function applies heuristics that favor free tiers when appropriate while maintaining quality standards.

**Latency** predicts turnaround time based on the `latency_p50_seconds` metric. The scoring maps sub-second responses to 1.0, while requests exceeding 60 seconds receive 0.2, with linear interpolation between these bounds.

**Continuity** ensures stylistic consistency by preferring providers already locked into the current production path. The `_compute_continuity` function boosts scores for providers used in recent operations, preventing jarring style shifts across sequences.

## How the Weighted Score Is Calculated

After computing individual dimension scores, the engine aggregates them using fixed weights defined in `ProviderScore.weighted_score` (lines 35-45). The weight distribution prioritizes task relevance and quality while balancing operational concerns:

| Dimension | Weight |
|-----------|--------|
| task_fit | 0.30 |
| output_quality | 0.20 |
| control | 0.15 |
| reliability | 0.15 |
| cost_efficiency | 0.10 |
| latency | 0.05 |
| continuity | 0.05 |

The `score_provider` function (lines 73-131) builds a `ProviderScore` instance for each candidate by invoking the individual dimension calculators. Subsequently, `rank_providers` (lines 33-41) sorts these scores in descending order, producing a ranked list that downstream selectors consume.

## How video_selector Uses the Scoring Engine

The `video_selector` tool in [`tools/video/video_selector.py`](https://github.com/calesthio/OpenMontage/blob/main/tools/video/video_selector.py) implements a five-stage pipeline that leverages the scoring engine to route video generation requests intelligently.

### Candidate Discovery and Context Preparation

First, `_providers()` gathers all registered tools with `capability="video_generation"`. Then `_prepare_task_context` calls `normalize_task_context` (lines 42-50) to transform loose user payloads into structured dictionaries required by the scorer, standardizing prompts, operations, and style parameters.

### Scoring and Provider Selection

Inside the `execute` method (lines 13-19), when the operation mode is not `"rank"`, the selector invokes:

```python
rankings = rank_providers(candidates, task_context)

```

If the user specifies a `preferred_provider`, the selector checks whether that provider’s score lies within `preferred_provider_gap` (default 0.15) of the top score (lines 28-30). When the preferred provider meets this threshold, it wins selection; otherwise, the highest-scored provider prevails (lines 21-33).

### Result Augmentation and Transparency

The selected tool’s `ProviderScore.explain()` method attaches human-readable reasoning to the final `ToolResult` (lines 55-57). For debugging or explicit ranking requests, `_serialize_rankings` (lines 68-82) exports the full provider ranking, enabling audit trails and optimization analysis.

## Practical Implementation Examples

The following example demonstrates manual ranking of video providers for a cinematic brief:

```python
from lib.scoring import rank_providers, normalize_task_context
from tools.video.video_selector import VideoSelector

selector = VideoSelector()
candidates = selector._providers()                      # all video tools

task_context = normalize_task_context(
    {},
    prompt="Create a cinematic trailer for a futuristic city",
    capability="video_generation",
    operation="text_to_video",
)

rankings = rank_providers(candidates, task_context)
for r in rankings[:3]:
    print(r.explain())

```

To use the selector in diagnostic "rank" mode without generating assets:

```python
result = selector.execute({
    "prompt": "A short explainer about AI safety",
    "operation": "rank",
    "target_operation": "text_to_video",
})
print(result.data["explanation"])   # human-readable reasons for top 5 providers

```

## Summary

- The **7-dimension provider scoring engine** in [`lib/scoring.py`](https://github.com/calesthio/OpenMontage/blob/main/lib/scoring.py) provides deterministic, explainable rankings across task fit, quality, control, reliability, cost, latency, and continuity dimensions.
- **Fixed weights** prioritize task fit (30%) and output quality (20%), ensuring relevance and fidelity remain primary selection criteria.
- **`video_selector`** automates candidate discovery, context normalization, and preference-aware selection through the `rank_providers` pipeline.
- **Transparency features** like `ProviderScore.explain()` and `_serialize_rankings` provide audit trails for every routing decision.
- The **preferred provider gap** (default 0.15) allows user preferences to override the top-ranked provider when quality degradation remains within acceptable bounds.

## Frequently Asked Questions

### What are the seven dimensions used in OpenMontage's provider scoring engine?

The seven dimensions are **task fit** (keyword relevance), **output quality** (historic fidelity or stability tier), **control** (creative parameter availability), **reliability** (success probability), **cost efficiency** (value-for-money), **latency** (expected turnaround time), and **continuity** (consistency with current production providers). Each returns a normalized 0-1 score, with weights aggregating them into a final ranking.

### How does video_selector handle user-preferred providers?

When a user specifies `preferred_provider` in the execution payload, `video_selector` compares that provider’s score against the top-ranked provider. If the gap falls within `preferred_provider_gap` (default 0.15), the preferred provider is selected despite not having the absolute highest score. If the gap exceeds this threshold, the engine defaults to the highest-scored provider to prevent significant quality degradation.

### Can the dimension weights be customized per project?

According to the source code in [`lib/scoring.py`](https://github.com/calesthio/OpenMontage/blob/main/lib/scoring.py), the weights are **hardcoded** in `ProviderScore.weighted_score` (lines 35-45) with fixed values for task fit (0.30), output quality (0.20), and the remaining five dimensions. The current implementation does not expose runtime weight customization, though individual dimension scores respond dynamically to task context and provider capabilities.

### How does the scoring engine ensure transparency in its decisions?

Every `ProviderScore` instance includes an `explain()` method that returns human-readable justifications for the computed scores. The `video_selector` attaches these explanations to `ToolResult` objects and can serialize full rankings via `_serialize_rankings`, allowing developers to inspect exactly why specific providers were chosen or rejected for any given task.