How OpenClaude Smart Auto-Routing Optimizes API Costs

OpenClaude’s Smart Auto-Routing is an optional CLI feature that classifies each user turn as simple or strong and routes trivial requests to a low-cost model while reserving expensive, capable models for complex queries, thereby lowering total token expenses without sacrificing answer quality.

Smart Auto-Routing in the Gitlawb/openclaude repository is implemented as a multi-stage pipeline that intercepts chat turns before they reach the LLM provider. By analyzing prompt characteristics and enforcing organizational policies, the system ensures that inexpensive models handle routine interactions while automatically escalating to stronger models when complexity or permission constraints demand it.

Configuration and Environment Setup

Users enable Smart Auto-Routing via the smartRouting block in ~/.openclaude.json or through environment variables. The configuration requires two distinct roles: a simpleModel for inexpensive inference and a strongModel for demanding tasks.

Environment variables provide startup defaults that override base settings:

export OPENCLAUDE_SMART_ROUTING=true
export OPENCLAUDE_SMART_ROUTING_SIMPLE=mini
export OPENCLAUDE_SMART_ROUTING_STRONG=main

According to the source code in src/services/api/smartRouting/settings.ts, these values are merged with user preferences at runtime. The resolveSmartRoutingConfig() function in src/services/api/smartRouting/resolveConfig.ts then normalizes this input—combining user settings, the current parent model, and permission modes—to produce a concrete SmartRoutingConfig object used for all subsequent routing decisions.

The Routing Decision Pipeline

When a user turn arrives, decideTurnModel() in src/services/api/smartRouting/index.ts executes a four-step resolution process:

1. Model String Resolution

The resolveSmartRoutingRoleModelString() function (lines 23-30 in src/services/api/smartRouting/index.ts) converts abstract agent model keys (e.g., mini) into concrete provider IDs (e.g., gpt-5-mini). If the key is already a bare model ID, it returns unchanged, ensuring the pipeline works with explicit provider identifiers.

2. Heuristic Classification

The core intelligence lives in src/services/api/smartModelRouting.ts. This heuristic classifier inspects four signal dimensions:

• Prompt length – Short queries typically indicate simple intent
• Code blocks – Presence of multi-line code suggests complex reasoning
• Planning keywords – Terms indicating architectural decisions or multi-step logic
• Session position – First turns often require more context establishment

If any heuristic suggests non-trivial work, the turn is marked strong; otherwise it receives a simple label. The classifier is deliberately conservative—uncertain cases default to strong, guaranteeing that the worst-case scenario is no cost savings rather than degraded output quality.

3. Allow-List Enforcement

After classification, decideTurnModel() (lines 96-133) validates the chosen model against organization-wide permissions via isModelAllowed(). If the simple model is disallowed, the system coerces the decision to the strong model. If both models are disallowed, Smart Auto-Routing is automatically disabled for the current session and the session ID is recorded in a bounded disabledSessions set (lines 65-81) to prevent repeated warning notices.

4. Fallback on Transport Errors

If a request to the simple model fails with a transport or server error (any status except 400, 401, or 403), the isRetryableRoutedModelError() function (lines 38-53) triggers a single retry using the strong model. Authentication and bad-request errors are not retried, preventing infinite loops on configuration mistakes.

Runtime Control and Cost Tracking

End-users interact with Smart Auto-Routing through slash commands. The /smartroute command enables, disables, or reconfigures roles mid-session:

# Enable routing

$ /smartroute on
Smart routing enabled: simple=mini, strong=main

# Change the simple model to a cheaper option

$ /smartroute simple tiny
Simple model set to "tiny"

# View current status

$ /smartroute
Smart routing: enabled
  simple model → mini (gpt-5-mini)
  strong model → main (gpt-5)

Each routed turn increments a per-session RoutingTally that tracks simple, strong, and escalations counts. The /cost command invokes formatRoutingSummary() (lines 55-110) to display:

$ /cost
Smart routing: 7 simple, 3 strong, 1 escalated to strong
Estimated (first-party reference pricing): simple turns use a model priced ~45% lower per input token.

This estimate references the internal MODEL_COSTS table. If either model lacks a known price entry, the savings line is suppressed to avoid misleading projections.

Programmatic Integration

Developers can leverage the routing logic outside the interactive CLI by importing the core decision engine:

import { decideTurnModel } from './src/services/api/smartRouting/index.js';
import { readSmartRouting } from './src/services/api/smartRouting/settings.js';

const decision = decideTurnModel({
  settings: readSmartRouting(),
  parentModel: 'gpt-5',
  input: {
    userText: 'list files',
    turnNumber: 4,
    // Additional RoutingInput fields...
  },
});

if (decision.routed) {
  console.log(`Routing to ${decision.complexity} model: ${decision.model}`);
}

Summary

• Smart Auto-Routing reduces API costs by directing simple turns to cheaper models while reserving expensive inference for complex queries.
• Configuration resides in ~/.openclaude.json or environment variables (OPENCLAUDE_SMART_ROUTING_*), processed by resolveSmartRoutingConfig().
• The heuristic classifier in src/services/api/smartModelRouting.ts analyzes prompt length, code presence, keywords, and turn position to label complexity.
• decideTurnModel() enforces allow-lists and falls back to the strong model if the simple model is unauthorized or encounters retryable errors.
• Session-scoped tallies and the /cost command provide transparency into routing decisions and estimated savings.

Frequently Asked Questions

How does OpenClaude determine if a turn is "simple" or "strong"?

OpenClaude uses a heuristic classifier defined in src/services/api/smartModelRouting.ts that evaluates prompt length, the presence of code blocks, planning-related keywords, and whether it is the first turn of the session. If any indicator suggests non-trivial work, the turn is classified as strong; otherwise it is marked simple. The system defaults to strong when uncertain to prevent quality degradation.

What happens if the simple model is not available or fails?

If the simple model is disallowed by organizational policy, decideTurnModel() automatically coerces the request to the strong model. If the simple model returns a transport or server error (excluding 400/401/403), the isRetryableRoutedModelError() function triggers a single retry on the strong model. If both models are disallowed, Smart Auto-Routing is disabled for that session.

Can I change models dynamically during a conversation?

Yes. The /smartroute command allows runtime reconfiguration. You can toggle routing on or off with /smartroute on|off, or switch the simple/strong models mid-session using /smartroute simple <model> or /smartroute strong <model>. Changes take effect immediately for subsequent turns.

How accurate is the cost savings estimate in the /cost command?

The estimate is informational and based on first-party reference pricing from the internal MODEL_COSTS table. It compares the input token pricing between your configured simple and strong models. If either model lacks a known price entry, the estimate is hidden. The figure does not reflect actual provider billing or output token costs, so real savings may vary based on your specific API agreement and token consumption patterns.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}