How the WorkWeave Router Chooses the Optimal AI Model Provider: A Deep Dive into the Selection Pipeline

The WorkWeave Router selects the optimal AI model provider through a five-stage pipeline that combines static catalog lookups, runtime provider filtering, and a learned HMM (Hierarchical-Multivariate-Model) policy engine to deterministically pick the first eligible arm that satisfies all constraints.

The WorkWeave Router is an open-source request routing layer for AI models that intelligently distributes traffic across multiple upstream providers. Unlike simple round-robin or random selection, the router implements a sophisticated, policy-aware decision pipeline that evaluates model ownership, deployment configuration, and learned behavioral patterns to route each request to the optimal provider.

The Five-Stage Selection Pipeline

When a request enters the system through the Route method in internal/router/router.go, the WorkWeave Router executes a deterministic sequence of filters to narrow the candidate set to a single provider.

Stage 1: Catalog Lookup for Model-to-Provider Binding

The router first consults the model catalog to resolve the requested model name to its canonical provider.

In internal/router/catalog/lookup.go, the catalog maintains a definitive mapping between model identifiers (e.g., claude-opus-4-8, gpt-4) and their owning providers (e.g., Anthropic, OpenAI, Google). When a router.Request arrives with a Model field populated, the resolver queries this catalog to determine the candidate provider.

If the requested model does not exist in the catalog, the pipeline terminates early. If it exists, the router passes the candidate provider to the next stage.

Stage 2: Enabled-Provider Filtering via Runtime Configuration

Before invoking the policy engine, the router filters candidates against the deployment's runtime constraints.

The router.Request struct carries an EnabledProviders field—a map[string]struct{} populated from the installation's configuration that specifies which providers are active for the current deployment. In internal/router/rl/rl.go, the system constructs a policy.Resolver that intersects the catalog-derived candidate providers with the EnabledProviders set.

If a model's canonical provider is not present in EnabledProviders, the resolver discards it, triggering a fallback to the next eligible model in the request's ranked fallback groups. This ensures that disabled or deprecated providers never receive traffic, even if they own the requested model in the catalog.

Stage 3: Policy-Level Arm Selection

With the filtered set of eligible providers identified, the router invokes the policy arm selector to prepare the selection context.

Located in internal/router/policy/arm_selector.go, the arm selector examines the request's ranked fallback groups and constructs the roster of available arms (provider-model pairs). An arm represents a concrete, routable target consisting of a specific provider and model combination.

The selector validates that at least one arm exists that satisfies the hard constraints. If the roster is empty or all arms violate policy constraints, the selector logs a warning and returns ErrNoEligibleArm, causing the router to fail gracefully.

Stage 4: HMM-Based Selection Logic

The Hierarchical-Multivariate-Model (HMM) selector performs the final optimization step, choosing the specific arm that best balances latency, cost, and availability.

In internal/router/hmm/selection/selector.go, the Select function receives:

• The roster of candidate arms
• The ranked fallback groups from the request
• A harness containing runtime telemetry
• The filtered candidate providers

The HMM selector iterates through the ranked groups in order, returning the first arm that satisfies the policy's probabilistic constraints. This learned model accounts for historical performance metrics, enabling the router to prefer providers with lower latency or higher reliability when multiple technically valid options exist.

Stage 5: Final Decision and Credential Injection

Once the HMM selector returns a policy.SelectionPick containing the Group and Arm, the router constructs a router.Decision struct.

This decision object contains:

• Provider: The selected upstream provider (e.g., "anthropic")
• Model: The specific model identifier
• CandidateProviders: The full set of providers considered during selection

The decision returns to the proxy service, which calls resolveAndInjectCredentials in internal/proxy/service.go. This function retrieves the appropriate API credentials for the selected provider, injects them into the request headers, and forwards the request to the upstream endpoint.

Implementation Example: Routing a Request

To leverage the WorkWeave Router in your application, construct a router.Request with your desired model and enabled providers, then invoke the Route method:

// Construct a request targeting Claude with Anthropic as the preferred provider
req := router.Request{
    Model: "claude-opus-4-8",
    EnabledProviders: map[string]struct{}{
        providers.ProviderAnthropic: {},
        providers.ProviderOpenAI:    {},
    },
}

// Execute the routing decision pipeline
decision, err := routerInstance.Route(ctx, req)
if err != nil {
    if errors.Is(err, router.ErrNoEligibleProvider) {
        log.Error("No provider available for the requested model")
        return
    }
    log.Error("Routing failed", "error", err)
    return
}

log.Info("Selected optimal provider",
    "provider", decision.Provider,
    "model", decision.Model,
    "candidates", decision.CandidateProviders,
)

For custom policy implementations, you can interact directly with the HMM selector interface:

// Custom selector implementation pattern (simplified)
func CustomSelector(roster *rosterdata.Roster) policy.ArmSelector {
    return func(ctx context.Context, input policy.SelectionInput) (policy.SelectionPick, error) {
        pick, ok := Select(roster, rankedGroups, input.Harness, candidates)
        if !ok {
            return policy.SelectionPick{}, ErrNoEligibleArm
        }
        return policy.SelectionPick{
            Group: pick.Group,
            Arm:   pick.Arm,
        }, nil
    }
}

Error Handling and Fallback Behavior

The WorkWeave Router implements graceful degradation when no optimal path exists.

If the policy arm selector cannot identify a valid arm, it returns ErrNoEligibleArm from internal/router/policy/arm_selector.go. If the higher-level routing logic determines that no providers are available for the requested model after filtering, it returns ErrNoEligibleProvider.

These errors prevent requests from reaching upstream providers in an invalid state, allowing client applications to implement retry logic or failover to alternative models. The router never defaults to a random provider; every selection must explicitly satisfy all catalog, configuration, and policy constraints.

Summary

• Catalog Lookup: The router resolves model names to canonical providers using internal/router/catalog/lookup.go before applying any runtime logic.
• Provider Filtering: The EnabledProviders set in router.Request ensures only deployment-approved providers are considered, enforced by the resolver in internal/router/rl/rl.go.
• HMM Optimization: The learned selector in internal/router/hmm/selection/selector.go picks the first arm that satisfies probabilistic constraints for latency and reliability.
• Deterministic Output: The final router.Decision struct contains the definitive Provider and Model, with credentials injected by internal/proxy/service.go before forwarding.

Frequently Asked Questions

What happens if the requested model's provider is disabled?

The router discards the disabled provider during the filtering stage in internal/router/rl/rl.go and attempts to route to the next eligible model in the request's ranked fallback groups. If no alternative models or providers satisfy the constraints, the router returns ErrNoEligibleProvider and the request fails gracefully without reaching upstream services.

How does the HMM selector determine the optimal provider?

The HMM (Hierarchical-Multivariate-Model) selector in internal/router/hmm/selection/selector.go evaluates arms using learned patterns from historical telemetry, including latency distributions and success rates. It iterates through ranked fallback groups and selects the first arm that meets the policy's probabilistic thresholds for performance and reliability, ensuring the choice is both valid and optimal.

Can I restrict routing to specific providers?

Yes. Populate the EnabledProviders field in the router.Request struct with a map containing only the providers you want to allow. The resolver in internal/router/rl/rl.go automatically filters out any providers not present in this set, preventing accidental routing to unauthorized or experimental endpoints.

Where is the final routing decision stored?

The router encapsulates the result in a router.Decision struct defined in internal/router/router.go. This struct contains the selected Provider, Model, and the full list of CandidateProviders considered during the pipeline, which the proxy service uses in internal/proxy/service.go to inject credentials and forward the request.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}