# OpenHuman Model Routing Strategies: Local vs Remote LLM Provider Selection

> Discover OpenHuman model routing strategies for local vs remote LLM provider selection. Learn how policy driven routing and health checks optimize task execution efficiently.

- Repository: [Tiny Humans/openhuman](https://github.com/tinyhumansai/openhuman)
- Tags: internals
- Published: 2026-05-12

---

**OpenHuman uses a policy-driven routing layer that classifies tasks by weight (Lightweight, Medium, Heavy), checks local AI health, and applies routing hints to decide whether to execute LLM calls locally via Ollama or remotely via the OpenHuman backend, with automatic fallback on quality failures.**

The `tinyhumansai/openhuman` repository implements a sophisticated model routing system that sits between your application code and concrete LLM providers. This routing layer intelligently distributes inference workloads across local Ollama-compatible servers and remote OpenHuman backends based on task characteristics, system health, and user-defined constraints.

## The Three Model Routing Strategies

OpenHuman employs three complementary **model routing strategies** to direct tasks to the appropriate LLM provider. Each strategy addresses different deployment scenarios and operational requirements.

### Intelligent Routing

**Intelligent routing** is the default strategy when `config.local_ai.runtime_enabled` is `true` and no explicit model routes are configured. This strategy uses the `policy::decide` function in [`src/openhuman/routing/policy.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/routing/policy.rs) to dynamically select between local and remote providers.

The system maps model hints (e.g., `"hint:reaction"`, `"hint:reasoning"`) to `TaskCategory` values—**Lightweight**, **Medium**, or **Heavy**—then evaluates local health status and `RoutingHints` to determine the primary target and optional fallback. Heavy tasks always route remotely unless privacy constraints force local execution, while lightweight tasks prefer local inference when health checks pass.

### Explicit Routing

**Explicit routing** bypasses the policy engine entirely when administrators define specific model routes in the configuration file. In [`src/openhuman/providers/ops.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/providers/ops.rs), the `create_routed_provider` function constructs a `RouterProvider` that maps specific hints directly to remote model names.

For example, a configuration entry might route `"hint:agentic"` directly to `gpt-4o-mini`, ensuring consistent provider selection regardless of task weight or local system status. This strategy is ideal when you require deterministic model selection for specific use cases.

### Remote-Only Fallback

**Remote-only fallback** activates when `config.local_ai.runtime_enabled` is `false` or when the health probe in [`src/openhuman/routing/health.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/routing/health.rs) determines that the local Ollama server is unreachable. In this mode, all requests route to the remote OpenHuman backend through the `OpenAiCompatibleProvider`, ensuring service continuity even when local infrastructure fails.

## The Routing Decision Pipeline

The routing layer processes every LLM request through a specific decision flow implemented across multiple source files.

### Task Classification

The pipeline begins in [`src/openhuman/routing/policy.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/routing/policy.rs), where the `classify` function converts model hints into task categories:

```rust
use openhuman::routing::policy::{classify, TaskCategory};

let category = classify("hint:reaction");
// Returns TaskCategory::Lightweight

```

This classification determines the baseline routing behavior before other factors intervene.

### Health Verification

Before selecting a local provider, the system executes a health check via [`src/openhuman/routing/health.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/routing/health.rs). This probe verifies that the local Ollama instance or OpenAI-compatible server is responsive and capable of accepting requests.

### Policy Evaluation

The `decide` function in [`src/openhuman/routing/policy.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/routing/policy.rs) evaluates `RoutingHints`—including `privacy_required`, `latency_budget`, and `cost_sensitivity`—to refine the routing target:

```rust
use openhuman::routing::policy::{
    decide, RoutingHints, RoutingTarget, TaskCategory, LatencyBudget, CostSensitivity,
};

let hints = RoutingHints {
    privacy_required: false,
    latency_budget: LatencyBudget::Low,
    cost_sensitivity: CostSensitivity::Normal,
};

let (primary, fallback) = decide(
    TaskCategory::Lightweight,
    "local-llama:7b",
    "remote-reasoning-v1",
    true,  // local health OK
    &hints,
);

```

### Execution with Quality Fallback

The `IntelligentRoutingProvider` in [`src/openhuman/routing/provider.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/routing/provider.rs) executes the primary request. If the local provider returns an error or the `quality::is_low_quality` heuristic judges the response insufficient, the system automatically retries against the fallback target—unless `privacy_required` is `true`, which suppresses remote calls entirely.

## Implementation Architecture

The routing system relies on several key components working together across the codebase.

### Provider Construction

In [`src/openhuman/providers/ops.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/providers/ops.rs), the `create_intelligent_routing_provider` function assembles the provider chain:

```rust
let provider = openhuman::providers::ops::create_intelligent_routing_provider(
    Some("https://api.openhuman.com"),
    Some("my-api-key"),
    &config,
    &ProviderRuntimeOptions::default(),
)?;

```

This function instantiates both a remote `OpenAiCompatibleProvider` for the OpenHuman backend and a local `OpenAiCompatibleProvider` for Ollama, wrapping them in an `IntelligentRoutingProvider` that handles the routing logic.

### Explicit Route Configuration

To use explicit routing, define `model_routes` in your configuration file and call `create_routed_provider`:

```toml

# config.toml

[[model_routes]]
hint = "hint:agentic"
model = "gpt-4o-mini"

```

```rust
let routed = openhuman::providers::ops::create_routed_provider(
    None,
    None,
    &reliability_cfg,
    &config.model_routes,
    "default-remote-model",
)?;

```

### Telemetry and Observability

Every routing decision emits a `RoutingRecord` via [`src/openhuman/routing/telemetry.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/routing/telemetry.rs), capturing the model hint, task category, selected provider, latency, fallback flags, and token usage. These records appear under the `"routing"` tracing target, enabling detailed audit trails of provider selection.

## Key Source Files

Understanding the complete routing implementation requires familiarity with these specific files:

- [`src/openhuman/routing/provider.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/routing/provider.rs) - Contains `IntelligentRoutingProvider`, which implements the `Provider` trait and handles fallback logic
- [`src/openhuman/routing/policy.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/routing/policy.rs) - Defines `TaskCategory`, `RoutingHints`, and the `decide` function for policy evaluation
- [`src/openhuman/routing/factory.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/routing/factory.rs) - Constructs routing providers from configuration objects
- [`src/openhuman/providers/ops.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/providers/ops.rs) - Public API for creating intelligent and routed providers
- [`src/openhuman/routing/telemetry.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/routing/telemetry.rs) - Emits routing decisions for observability
- [`src/openhuman/routing/health.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/routing/health.rs) - Performs health probes on local AI runtimes
- [`src/openhuman/routing/quality.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/routing/quality.rs) - Implements heuristics for detecting low-quality local responses

## Summary

- **OpenHuman employs policy-driven routing** that weighs task characteristics against system health and user constraints to select between local and remote LLM providers.
- **Three strategies cover all scenarios**: intelligent routing for dynamic selection, explicit routing for deterministic model mapping, and remote-only fallback for high-availability requirements.
- **Automatic quality fallback** allows the system to retry failed or low-quality local inferences against remote backends unless privacy requirements prohibit external calls.
- **Comprehensive telemetry** via `RoutingRecord` events enables full observability of routing decisions under the `"routing"` tracing target.
- **Configuration-driven architecture** lets administrators define specific model routes in TOML while maintaining intelligent defaults for unspecified hints.

## Frequently Asked Questions

### How does OpenHuman decide between local and remote LLM providers?

OpenHuman uses the `decide` function in [`src/openhuman/routing/policy.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/routing/policy.rs) to evaluate task categories (Lightweight, Medium, Heavy), local health status, and routing hints. Heavy tasks route remotely by default, lightweight tasks prefer local execution when healthy, and medium tasks consider latency budgets and cost sensitivity to determine the optimal provider.

### What happens if the local Ollama server fails during a request?

If the primary target is local and the call errors or produces low-quality output (as determined by `quality::is_low_quality` in [`src/openhuman/routing/quality.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/routing/quality.rs)), the `IntelligentRoutingProvider` automatically retries the request against the remote fallback target. This transparent fallback ensures request completion unless `privacy_required` is set to `true` in the routing hints.

### Can I force specific model hints to use specific providers?

Yes, by defining `model_routes` in your configuration file and using `create_routed_provider` from [`src/openhuman/providers/ops.rs`](https://github.com/tinyhumansai/openhuman/blob/main/src/openhuman/providers/ops.rs), you can map specific hints directly to concrete models. This explicit routing bypasses the policy engine entirely, ensuring deterministic provider selection for critical workflows.

### What routing information is available for monitoring?

The `routing::telemetry` module emits `RoutingRecord` events for every routing decision, including the model hint, `TaskCategory`, selected provider, latency metrics, fallback flags, and token usage. These records are emitted under the `"routing"` tracing target, compatible with standard Rust logging infrastructure for audit and monitoring purposes.