How OmniRoute's Token Health Check Scheduler Refreshes OAuth Tokens

OmniRoute proactively refreshes OAuth tokens using a background scheduler that runs every 60 seconds, batches connections for efficiency, and implements circuit-breaker back‑off to prevent provider rate limits.

OmniRoute maintains uninterrupted provider access through an intelligent token health check scheduler that monitors OAuth connections in real time. This background service, implemented in src/lib/tokenHealthCheck.ts, works in tandem with the token refresh orchestrator at open-sse/services/tokenRefresh.ts to detect expiring credentials and rotate them before they become invalid.

Scheduler Initialization and Tick Loop

The token health check scheduler initializes automatically when the module loads. The initTokenHealthCheck() function (defined at the bottom of src/lib/tokenHealthCheck.ts) creates a singleton instance guarded by globalThis.__omnirouteTokenHC to survive hot‑reloads in development environments.

The scheduler begins with a one‑off timeout of 10 seconds to allow the application to stabilize, then registers a repeating setInterval using TICK_MS = 60 000 ms (one minute). This cadence ensures frequent enough checks to catch tokens approaching expiry while minimizing CPU and database load.

// Manually start the health‑check (useful in tests)
import { initTokenHealthCheck, stopTokenHealthCheck } from "@/lib/tokenHealthCheck";

initTokenHealthCheck();   // starts the 60 s tick sweep
// …
stopTokenHealthCheck();   // clean shutdown

The Sweep Process: Batching and Jitter

Every tick triggers the sweep() function (lines 25–80 in src/lib/tokenHealthCheck.ts), which queries all active OAuth connections using getProviderConnections({ authType: "oauth" }).

To prevent thundering herds against the database or external providers, the scheduler processes connections in configurable batches controlled by HEALTHCHECK_BATCH_SIZE (default 20). Between batches, it optionally inserts a jittered delay defined by HEALTHCHECK_STAGGER_MS to smooth out request distribution.

Per-Connection Health Verification

For each connection, checkConnection(conn) applies a series of filters and checks to determine if a refresh is necessary.

Skip Conditions

The scheduler immediately skips connections that are:

• Disabled or in a terminal state
• Belonging to providers listed in OMNIROUTE_HEALTHCHECK_SKIP_PROVIDERS
• Configured with a zero healthCheckInterval

Expiry Detection Logic

For eligible connections, the scheduler calculates whether the token is near expiry using TOKEN_EXPIRY_BUFFER = 5 minutes. If the token expires within this window, or if the provider requires fixed‑interval refresh (non‑rotating providers), the scheduler flags the connection for rotation.

When rotation is needed, checkConnection logs the intent and delegates to getAccessToken() from open-sse/services/tokenRefresh.ts.

// Force a refresh for a single connection (e.g. from an admin UI)
import { checkConnection } from "@/lib/tokenHealthCheck";

const conn = await getCachedProviderConnectionById("connection-id");
await checkConnection(conn);   // runs the same logic the scheduler uses

Token Refresh Orchestration

The getAccessToken() function in open-sse/services/tokenRefresh.ts (lines 44–88) handles the actual OAuth handshake with several resilience mechanisms.

Deduplication and Staleness Guards

Each connection maintains a per‑connection mutex to deduplicate concurrent refresh attempts. Before executing the refresh, the service performs a staleness check against the database to avoid using a refresh token that another process already invalidated, which would trigger "refresh_token_reused" errors.

The implementation uses a compare‑and‑swap (CAS) guard: if another process rotated the refresh token between the read and write, the database update is skipped to prevent overwriting the newer credentials.

Provider-Specific Handlers

OmniRoute routes refresh requests to provider‑specific implementations (such as refreshCodexToken, refreshGoogleToken, etc.) or falls back to the generic refreshAccessToken for standard OAuth2 flows.

// Retrieve a refreshed access token for a provider (used internally)
import { getAccessToken } from "@omniroute/open-sse/services/tokenRefresh";

const result = await getAccessToken(
  "google",
  { refreshToken: conn.refreshToken, connectionId: conn.id },
  console,                // logger
  null,                   // optional proxy config
);

if (result?.accessToken) {
  console.log("New token:", result.accessToken);
}

Circuit Breaker and Back-off Strategy

When a refresh fails, buildRefreshFailureUpdate() records the failure and computes an exponential back‑off ranging from REFRESH_CIRCUIT_BASE_MIN to REFRESH_CIRCUIT_MAX_MIN. The isInRefreshBackoff() function checks this state during subsequent sweeps, causing the scheduler to skip the connection until the back‑off period expires. This prevents hammering providers with invalid or expired credentials.

Special Provider Handling

Rotating Token Providers

Providers like Codex, OpenAI, and Kimi‑coding are marked in ROTATING_REFRESH_PROVIDERS. For these, the scheduler refreshes tokens only when they are truly about to expire (within the 5‑minute buffer), avoiding unnecessary rotations that could invalidate sibling accounts sharing the same refresh token.

GitHub Copilot Sub-tokens

Connections possessing only a GitHub access token (without a refresh token) route through refreshCopilotToken and, when necessary, refreshGithubCopilotCopilotSubTokenIfNeeded in src/lib/tokenHealthCheckCopilot.ts. This specialized flow handles GitHub Copilot’s unique authentication model where sub‑tokens require separate lifecycle management.

Result Persistence and Error Handling

Upon successful refresh, the health check updates the connection row via updateProviderConnection(), persisting the new accessToken, refreshToken, expiry timestamps, and clearing any circuit‑breaker state via clearRefreshCircuit().

If the refresh encounters an unrecoverable error (detected by isUnrecoverableRefreshError), the connection is marked with testStatus: "expired" and a detailed error is logged for administrative review.

Summary

• OmniRoute's token health check scheduler runs every 60 seconds in src/lib/tokenHealthCheck.ts to proactively refresh OAuth tokens before expiry.
• Batch processing with jitter prevents database and provider overload by processing connections in configurable chunks with optional delays.
• Circuit-breaker logic implements exponential back‑off for failed refreshes to avoid rate‑limiting and credential lockouts.
• Provider-specific handling accommodates rotating token providers and special cases like GitHub Copilot sub‑tokens.
• Deduplication and CAS guards ensure atomic token updates even under concurrent access across multiple processes.

Frequently Asked Questions

How often does OmniRoute check token health?

The scheduler runs a full sweep every 60 seconds (TICK_MS = 60 000 ms), with an initial 10‑second delay on startup to allow the application to stabilize.

What happens if a token refresh fails repeatedly?

The scheduler activates a circuit breaker that applies exponential back‑off ranging from REFRESH_CIRCUIT_BASE_MIN to REFRESH_CIRCUIT_MAX_MIN. During this period, isInRefreshBackoff() prevents further refresh attempts for that connection.

Why does OmniRoute batch token health checks?

Batching (default size of 20 connections) combined with optional jittered delays (HEALTHCHECK_STAGGER_MS) prevents thundering herds against the database and upstream OAuth providers, ensuring sustainable resource usage at scale.

How does OmniRoute handle rotating refresh tokens?

For providers in ROTATING_REFRESH_PROVIDERS (such as OpenAI and Codex), the scheduler refreshes only when the token is within 5 minutes of expiry. This conservative approach prevents unnecessary rotations that could invalidate refresh tokens shared across sibling connections.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}