# How FreeLLMAPI Handles HTTP Requests Through Upstream Proxies

> Learn how FreeLLMAPI routes HTTP requests through upstream proxies using a lazy-loaded dispatcher. Supports HTTP/HTTPS and SOCKS with per-platform bypass.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: internals
- Published: 2026-06-30

---

**FreeLLMAPI routes outbound HTTP requests through configurable upstream proxies using a lazy-loaded dispatcher system that supports HTTP/HTTPS via undici and SOCKS via socks-proxy-agent, with per-platform bypass capabilities and automatic fallback to direct fetch.**

The `tashfeenahmed/freellmapi` repository implements a self-contained proxy layer that intercepts every outbound HTTP call to determine whether it should travel through an upstream proxy or connect directly. This architecture ensures that API providers like OpenAI, Anthropic, and Google can be routed through corporate proxies or privacy-focused SOCKS tunnels while maintaining granular control over which platforms bypass the proxy entirely.

## Configuration and Runtime State

The proxy system initializes its configuration from the database at startup, reading the **proxy URL**, **enabled flag**, and **per-platform bypass list**. These values can be overridden by the `PROXY_URL` environment variable for containerized deployments.

In [`server/src/lib/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/proxy.ts), the functions `applyProxyUrl`, `applyProxyEnabled`, and `applyProxyBypass` (lines 39-79) handle this initialization. They are invoked once when the server boots and again whenever the administrator updates settings via `PUT /api/settings/proxy`. This ensures the proxy state remains consistent across the application lifecycle without requiring restarts.

```typescript
// Configuration structure managed by the proxy layer
interface ProxyConfig {
  proxyUrl: string | null;
  enabled: boolean;
  bypassPlatforms: Set<string>; // e.g., {"google", "anthropic"}
}

```

## Bypass Logic and Per-Platform Filtering

Before any network request executes, the `shouldBypassProxy` function (lines 88-96) evaluates whether the proxy should be skipped. The logic checks two conditions: whether the proxy is globally disabled, or whether the requested **platform** (such as "google" or "anthropic") appears in the bypass set.

If either condition is true, the request flows directly to the native `fetch` implementation, bypassing all proxy overhead. This allows administrators to route traffic for expensive or sensitive providers through different network paths while sending other traffic directly.

## Lazy Dispatcher Creation and Caching

To avoid loading heavy native dependencies when no proxy is configured, FreeLLMAPI implements **lazy dispatcher loading** in `resolveDispatcher` (lines 4-22). The first time a proxied request occurs, the system dynamically imports the appropriate agent library:

- **HTTP/HTTPS proxies**: Loads `undici`'s `ProxyAgent` via `loadHttpProxyAgent`
- **SOCKS proxies**: Loads `socks-proxy-agent` via `loadSocksAgent`

Once created, the dispatcher instance is cached for **30 seconds** (`CACHE_TTL_MS`) as implemented in lines 30-38 and 102-130. Subsequent requests reuse the same agent instance, eliminating the performance cost of repeated imports and object instantiation during high-throughput scenarios.

## Request Routing Implementation

The `proxyFetch(url, init?, platform?)` function serves as a drop-in replacement for the native `fetch` API across the entire codebase. Located in [`server/src/lib/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/proxy.ts) (lines 220-248), it implements a decision tree that determines the transport mechanism:

1. **Bypass triggered** → Direct `fetch` with no proxy
2. **No dispatcher available** (missing URL or creation failure) → Direct `fetch` with graceful fallback
3. **SOCKS dispatcher active** → Routes through `socksFetch` helper (lines 138-176) using Node.js native `http`/`https` modules with the SOCKS agent
4. **HTTP/HTTPS dispatcher active** → Standard `fetch` with the undici `dispatcher` option attached

```typescript
// Example: Provider implementation respecting upstream proxy configuration
import { proxyFetch } from '../lib/proxy.js';

async function callAnthropic(endpoint: string, body: any) {
  const res = await proxyFetch(
    `https://api.anthropic.com/v1/${endpoint}`,
    {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(body),
    },
    'anthropic' // Platform identifier for bypass evaluation
  );
  return await res.json();
}

```

## Error Handling and Fallback Behavior

If dispatcher creation throws an exception—such as when the proxy URL is malformed or the upstream proxy refuses connections—the error is logged and the request automatically falls back to a direct fetch (lines 30-35). This **fail-open** design ensures that proxy misconfigurations never block critical API functionality, allowing the application to continue operating even when the proxy layer encounters issues.

## Integration Across Services

All provider services throughout the codebase import `proxyFetch` instead of the native `fetch` global. For example, the media service ([`server/src/services/media.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/media.ts), lines 12-13) uses this function when downloading remote assets, ensuring that image or document retrieval respects the upstream proxy settings. Similarly, embedding services and LLM provider adapters (located in `server/src/services/providers/`) route their outbound traffic through this centralized proxy layer.

Runtime visibility into the proxy state is available via `isProxyActive()` (lines 250-258), which reports whether a proxy URL is set and enabled without triggering expensive dispatcher loading. The administrative dashboard uses this function to display the "Active" status badge.

```typescript
// Updating proxy configuration via the REST API
// PUT /api/settings/proxy (handled in server/src/routes/settings.ts)
{
  "proxyUrl": "http://myproxy.example.com:3128",
  "enabled": true,
  "bypassPlatforms": "google,anthropic"
}

```

## Summary

- **Configuration persistence**: Proxy settings are stored in the database and applied via `applyProxyUrl`, `applyProxyEnabled`, and `applyProxyBypass` in [`server/src/lib/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/proxy.ts), with optional `PROXY_URL` environment variable override.
- **Selective bypass**: The `shouldBypassProxy` function checks global enablement flags and per-platform exemptions before routing traffic.
- **Lazy loading**: `resolveDispatcher` defers importing `undici` or `socks-proxy-agent` until the first proxied request, reducing startup overhead.
- **Dispatcher caching**: Agent instances are cached for 30 seconds to optimize connection reuse and avoid repeated instantiation costs.
- **Transport flexibility**: `proxyFetch` handles HTTP/HTTPS via undici's `ProxyAgent` and SOCKS via `socks-proxy-agent`, with automatic fallback to direct fetch on errors.
- **Universal integration**: All outbound calls throughout the provider services and media handlers use `proxyFetch`, ensuring consistent proxy compliance.

## Frequently Asked Questions

### How does FreeLLMAPI determine whether to use a proxy for a specific request?

FreeLLMAPI checks the `shouldBypassProxy` function in [`server/src/lib/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/proxy.ts) (lines 88-96) before each outbound call. This function evaluates whether the proxy is globally disabled or whether the specific platform identifier (such as "anthropic" or "google") exists in the bypass list configured via the `bypassPlatforms` setting. If neither condition is met, the request routes through the upstream proxy.

### What happens if the configured proxy URL is invalid or unreachable?

The proxy layer implements a fail-open strategy. If `resolveDispatcher` throws an error while creating the agent—due to malformed URLs, network issues, or missing dependencies—the error is logged and `proxyFetch` automatically falls back to a direct fetch call (lines 30-35). This ensures that API requests continue to function even when the proxy configuration is broken.

### Does FreeLLMAPI support both HTTP and SOCKS proxies?

Yes. The system dynamically loads the appropriate agent based on the proxy URL scheme. For `http://` and `https://` URLs, it loads `undici`'s `ProxyAgent`. For `socks://`, `socks4://`, or `socks5://` URLs, it loads `socks-proxy-agent`. This selection occurs lazily on the first request requiring proxy routing, as implemented in lines 4-22 of [`server/src/lib/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/proxy.ts).

### How can I update proxy settings without restarting the server?

Send a `PUT` request to `/api/settings/proxy` (handled in [`server/src/routes/settings.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/settings.ts)) with a JSON payload containing `proxyUrl`, `enabled`, and `bypassPlatforms`. This triggers the `applyProxyUrl`, `applyProxyEnabled`, and `applyProxyBypass` functions to update the runtime configuration immediately, clearing the dispatcher cache and applying new settings without requiring a server restart.