How to Configure Proxy Settings for Outbound Requests in FreeLLMAPI

FreeLLMAPI routes all outbound LLM API calls through a configurable proxy layer that supports HTTP, HTTPS, and SOCKS5 protocols, manageable via environment variables at startup or REST API calls at runtime without requiring a restart.

The tashfeenahmed/freellmapi repository implements a flexible proxy system for all outbound requests to LLM providers. Whether you need to route traffic through a corporate firewall, anonymize requests via a SOCKS tunnel, or temporarily bypass specific providers from proxy routing, FreeLLMAPI provides both startup-time environment configuration and runtime API controls. This guide covers the complete proxy architecture, configuration methods, and practical implementation examples based on the source code.

Understanding the Proxy Architecture

Core Proxy Module (server/src/lib/proxy.ts)

The proxy functionality centers on server/src/lib/proxy.ts, which implements a thin, lazy-loaded wrapper around undici's ProxyAgent (for HTTP/HTTPS) and socks-proxy-agent (for SOCKS protocols). This module maintains three runtime globals that control all outbound network behavior:

• _proxyUrl – The destination proxy URL (e.g., http://proxy.company.com:8080 or socks5://127.0.0.1:1080).
• _proxyEnabled – Boolean toggle that activates or deactivates proxy usage globally.
• _bypassPlatforms – A set of provider identifiers (such as openai or anthropic) that skip the proxy even when enabled.

The module exposes helper functions including getProxyUrl(), isProxyActive(), applyProxyUrl(), applyProxyEnabled(), and applyProxyBypass() that provider adapters call before initiating outbound requests. When active, the system automatically wraps fetch requests through proxyFetch, ensuring consistent application of proxy settings across all LLM provider integrations.

Settings API Endpoint (server/src/routes/settings.ts)

Runtime configuration is exposed through the REST endpoint /api/settings/proxy defined in server/src/routes/settings.ts. This endpoint accepts GET requests to retrieve current settings and PUT requests to update them, with validation logic ensuring proxyUrl values conform to valid URL formats before persistence to the SQLite settings table.

Configuration Methods

Environment Variables for Startup Configuration

For containerized deployments or immutable infrastructure, FreeLLMAPI reads proxy settings from environment variables at startup. These variables are documented in the repository's .env.example file:

PROXY_URL=          # e.g., http://proxy.mycompany.com:3128 or socks5://proxy.example.com:1080

PROXY_ENABLED=1     # 1 = enabled, 0 = disabled

PROXY_BYPASS=       # comma-separated list, e.g., openai,anthropic

When present, these variables initialize the proxy state during the application bootstrap phase through the applyProxyUrl(), applyProxyEnabled(), and applyProxyBypass() functions in server/src/lib/proxy.ts.

Runtime Configuration via REST API

For dynamic environments, modify proxy behavior without restarting the service using the settings API. All changes take effect immediately on the next outbound request.

Step-by-Step Configuration Examples

Checking Current Proxy Status

Inspect the active configuration using a simple GET request:

curl -s http://localhost:3000/api/settings/proxy | jq .

This returns JSON containing the current proxyUrl, enabled boolean, and bypassPlatforms array.

Enabling an HTTP or SOCKS5 Proxy

Activate proxy routing with a PUT request to the settings endpoint:

curl -X PUT http://localhost:3000/api/settings/proxy \
     -H "Content-Type: application/json" \
     -d '{"proxyUrl":"http://proxy.mycompany.com:8080","enabled":true}'

For SOCKS5 tunnels, simply change the URL scheme:

curl -X PUT http://localhost:3000/api/settings/proxy \
     -H "Content-Type: application/json" \
     -d '{"proxyUrl":"socks5://127.0.0.1:1080","enabled":true}'

Bypassing Specific Providers

Route traffic directly to specific providers while sending others through the proxy using the bypassPlatforms parameter:

curl -X PUT http://localhost:3000/api/settings/proxy \
     -H "Content-Type: application/json" \
     -d '{"proxyUrl":"http://proxy.internal:8080","enabled":true,"bypassPlatforms":"openai,anthropic"}'

Programmatic Configuration with Node.js

Update the proxy configuration from within your application code:

async function configureProxy(url, enabled = true, bypass = '') {
  const res = await fetch('http://localhost:3000/api/settings/proxy', {
    method: 'PUT',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      proxyUrl: url,
      enabled,
      bypassPlatforms: bypass
    })
  });
  
  if (!res.ok) throw new Error(`Proxy update failed: ${await res.text()}`);
  console.log('Proxy configuration updated successfully');
}

// Configure SOCKS5 with OpenAI bypass
await configureProxy('socks5://proxy.example.com:1080', true, 'openai');

Docker Deployment with Proxy

Configure proxy settings at container startup using Docker Compose:

services:
  llmapi:
    image: tashfeenahmed/freellmapi:latest
    environment:
      - PROXY_URL=http://proxy.internal:8080
      - PROXY_ENABLED=1
      - PROXY_BYPASS=openai,anthropic
    ports:
      - "3000:3000"

How Provider Requests Use the Proxy

Provider adapters in server/src/providers/base.ts import the proxyFetch utility from server/src/lib/proxy.ts rather than using native fetch directly. This ensures that every outbound request to LLM APIs automatically respects the current proxy state, including enabled status and platform-specific bypass rules. The lazy-loading mechanism defers agent initialization until the first proxied request, minimizing startup overhead when proxy features are unused.

Summary

• FreeLLMAPI centralizes outbound request routing through server/src/lib/proxy.ts, supporting HTTP, HTTPS, and SOCKS5 proxy protocols.
• Three configuration layers control behavior: the _proxyUrl destination, _proxyEnabled global toggle, and _bypassPlatforms provider exclusions.
• Environment variables (PROXY_URL, PROXY_ENABLED, PROXY_BYPASS) initialize settings at startup for containerized deployments.
• REST API endpoint /api/settings/proxy enables runtime modification without service restarts, with changes applying immediately to subsequent requests.
• Provider adapters automatically respect proxy settings through the proxyFetch abstraction in server/src/providers/base.ts.

Frequently Asked Questions

Does FreeLLMAPI support SOCKS5 proxies?

Yes. The proxy module in server/src/lib/proxy.ts explicitly supports SOCKS5 endpoints via the socks-proxy-agent library. Configure SOCKS5 proxies using the standard URL format socks5://host:port in either the PROXY_URL environment variable or the REST API proxyUrl field.

Can I disable the proxy for specific LLM providers?

Yes. Use the bypassPlatforms setting (or PROXY_BYPASS environment variable) to provide a comma-separated list of provider identifiers such as openai, anthropic, or google. Requests to these providers will bypass the proxy even when the global proxy is enabled, as enforced by the _bypassPlatforms Set in the proxy module.

Do proxy changes require a server restart?

No. Changes made via the /api/settings/proxy endpoint take effect immediately for all subsequent outbound requests. The proxy module maintains runtime globals that provider adapters check on each request, allowing dynamic reconfiguration without application downtime.

How do I troubleshoot proxy connection issues?

First verify the current configuration using GET /api/settings/proxy to confirm the URL and enabled status. Check that your proxy URL includes the correct scheme (http://, https://, or socks5://). If specific providers fail while others succeed, review the bypassPlatforms list to ensure traffic isn't being inadvertently routed through the proxy for providers that block known proxy IPs. Finally, inspect the provider-specific implementations in server/src/providers/base.ts to confirm they utilize the proxyFetch utility rather than direct fetch calls.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}