How FreeLLMAPI Handles HTTP Requests Through Upstream Proxies

FreeLLMAPI routes outbound HTTP requests through configurable upstream proxies using a lazy-loaded dispatcher system that supports HTTP/HTTPS via undici and SOCKS via socks-proxy-agent, with per-platform bypass capabilities and automatic fallback to direct fetch.

The tashfeenahmed/freellmapi repository implements a self-contained proxy layer that intercepts every outbound HTTP call to determine whether it should travel through an upstream proxy or connect directly. This architecture ensures that API providers like OpenAI, Anthropic, and Google can be routed through corporate proxies or privacy-focused SOCKS tunnels while maintaining granular control over which platforms bypass the proxy entirely.

Configuration and Runtime State

The proxy system initializes its configuration from the database at startup, reading the proxy URL, enabled flag, and per-platform bypass list. These values can be overridden by the PROXY_URL environment variable for containerized deployments.

In server/src/lib/proxy.ts, the functions applyProxyUrl, applyProxyEnabled, and applyProxyBypass (lines 39-79) handle this initialization. They are invoked once when the server boots and again whenever the administrator updates settings via PUT /api/settings/proxy. This ensures the proxy state remains consistent across the application lifecycle without requiring restarts.

// Configuration structure managed by the proxy layer
interface ProxyConfig {
  proxyUrl: string | null;
  enabled: boolean;
  bypassPlatforms: Set<string>; // e.g., {"google", "anthropic"}
}

Bypass Logic and Per-Platform Filtering

Before any network request executes, the shouldBypassProxy function (lines 88-96) evaluates whether the proxy should be skipped. The logic checks two conditions: whether the proxy is globally disabled, or whether the requested platform (such as "google" or "anthropic") appears in the bypass set.

If either condition is true, the request flows directly to the native fetch implementation, bypassing all proxy overhead. This allows administrators to route traffic for expensive or sensitive providers through different network paths while sending other traffic directly.

Lazy Dispatcher Creation and Caching

To avoid loading heavy native dependencies when no proxy is configured, FreeLLMAPI implements lazy dispatcher loading in resolveDispatcher (lines 4-22). The first time a proxied request occurs, the system dynamically imports the appropriate agent library:

• HTTP/HTTPS proxies: Loads undici's ProxyAgent via loadHttpProxyAgent
• SOCKS proxies: Loads socks-proxy-agent via loadSocksAgent

Once created, the dispatcher instance is cached for 30 seconds (CACHE_TTL_MS) as implemented in lines 30-38 and 102-130. Subsequent requests reuse the same agent instance, eliminating the performance cost of repeated imports and object instantiation during high-throughput scenarios.

Request Routing Implementation

The proxyFetch(url, init?, platform?) function serves as a drop-in replacement for the native fetch API across the entire codebase. Located in server/src/lib/proxy.ts (lines 220-248), it implements a decision tree that determines the transport mechanism:

1. Bypass triggered → Direct fetch with no proxy
2. No dispatcher available (missing URL or creation failure) → Direct fetch with graceful fallback
3. SOCKS dispatcher active → Routes through socksFetch helper (lines 138-176) using Node.js native http/https modules with the SOCKS agent
4. HTTP/HTTPS dispatcher active → Standard fetch with the undici dispatcher option attached

// Example: Provider implementation respecting upstream proxy configuration
import { proxyFetch } from '../lib/proxy.js';

async function callAnthropic(endpoint: string, body: any) {
  const res = await proxyFetch(
    `https://api.anthropic.com/v1/${endpoint}`,
    {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(body),
    },
    'anthropic' // Platform identifier for bypass evaluation
  );
  return await res.json();
}

Error Handling and Fallback Behavior

If dispatcher creation throws an exception—such as when the proxy URL is malformed or the upstream proxy refuses connections—the error is logged and the request automatically falls back to a direct fetch (lines 30-35). This fail-open design ensures that proxy misconfigurations never block critical API functionality, allowing the application to continue operating even when the proxy layer encounters issues.

Integration Across Services

All provider services throughout the codebase import proxyFetch instead of the native fetch global. For example, the media service (server/src/services/media.ts, lines 12-13) uses this function when downloading remote assets, ensuring that image or document retrieval respects the upstream proxy settings. Similarly, embedding services and LLM provider adapters (located in server/src/services/providers/) route their outbound traffic through this centralized proxy layer.

Runtime visibility into the proxy state is available via isProxyActive() (lines 250-258), which reports whether a proxy URL is set and enabled without triggering expensive dispatcher loading. The administrative dashboard uses this function to display the "Active" status badge.

// Updating proxy configuration via the REST API
// PUT /api/settings/proxy (handled in server/src/routes/settings.ts)
{
  "proxyUrl": "http://myproxy.example.com:3128",
  "enabled": true,
  "bypassPlatforms": "google,anthropic"
}

Summary

• Configuration persistence: Proxy settings are stored in the database and applied via applyProxyUrl, applyProxyEnabled, and applyProxyBypass in server/src/lib/proxy.ts, with optional PROXY_URL environment variable override.
• Selective bypass: The shouldBypassProxy function checks global enablement flags and per-platform exemptions before routing traffic.
• Lazy loading: resolveDispatcher defers importing undici or socks-proxy-agent until the first proxied request, reducing startup overhead.
• Dispatcher caching: Agent instances are cached for 30 seconds to optimize connection reuse and avoid repeated instantiation costs.
• Transport flexibility: proxyFetch handles HTTP/HTTPS via undici's ProxyAgent and SOCKS via socks-proxy-agent, with automatic fallback to direct fetch on errors.
• Universal integration: All outbound calls throughout the provider services and media handlers use proxyFetch, ensuring consistent proxy compliance.

Frequently Asked Questions

How does FreeLLMAPI determine whether to use a proxy for a specific request?

FreeLLMAPI checks the shouldBypassProxy function in server/src/lib/proxy.ts (lines 88-96) before each outbound call. This function evaluates whether the proxy is globally disabled or whether the specific platform identifier (such as "anthropic" or "google") exists in the bypass list configured via the bypassPlatforms setting. If neither condition is met, the request routes through the upstream proxy.

What happens if the configured proxy URL is invalid or unreachable?

The proxy layer implements a fail-open strategy. If resolveDispatcher throws an error while creating the agent—due to malformed URLs, network issues, or missing dependencies—the error is logged and proxyFetch automatically falls back to a direct fetch call (lines 30-35). This ensures that API requests continue to function even when the proxy configuration is broken.

Does FreeLLMAPI support both HTTP and SOCKS proxies?

Yes. The system dynamically loads the appropriate agent based on the proxy URL scheme. For http:// and https:// URLs, it loads undici's ProxyAgent. For socks://, socks4://, or socks5:// URLs, it loads socks-proxy-agent. This selection occurs lazily on the first request requiring proxy routing, as implemented in lines 4-22 of server/src/lib/proxy.ts.

How can I update proxy settings without restarting the server?

Send a PUT request to /api/settings/proxy (handled in server/src/routes/settings.ts) with a JSON payload containing proxyUrl, enabled, and bypassPlatforms. This triggers the applyProxyUrl, applyProxyEnabled, and applyProxyBypass functions to update the runtime configuration immediately, clearing the dispatcher cache and applying new settings without requiring a server restart.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}