How to Connect FreeLLMAPI to Local LLMs Running Ollama or LM Studio

Yes, FreeLLMAPI includes a built‑in Ollama emulation layer that proxies any OpenAI‑compatible local server—including Ollama, LM Studio, llama.cpp, and vLLM—through a unified API surface.

FreeLLMAPI bridges the gap between local inference engines and standard client libraries by presenting your self‑hosted models as first‑class endpoints. When enabled, the gateway detects local instances, normalizes model identifiers, and translates between OpenAI‑style and Ollama‑native wire formats, allowing you to use the same SDKs and CLI tools you already know.

Understanding the Ollama Emulation Architecture

The emulation system is opt‑in and disabled by default. Once activated via the Key → Agent UI or the --ollama-emulation CLI flag, FreeLLMAPI establishes a transparent bridge between client applications and your local inference server.

At its core, the architecture performs four critical functions:

• Local endpoint detection – Identifies traffic destined for localhost or private subnets and exempts it from cloud‑style quota enforcement.
• Model name normalization – Strips Ollama‑specific tags (e.g., :latest) and maps proprietary identifier fields to canonical model names.
• Request protocol translation – Rewrites OpenAI‑compatible JSON payloads to match Ollama’s expected schema and vice‑versa.
• Authentication middleware – Optionally secures local endpoints with per‑token URL patterns while preserving the standard REST interface.

How FreeLLMAPI Detects and Routes to Local Endpoints

URL Detection and Trust Rules

In server/src/lib/url-guard.ts, FreeLLMAPI implements a hostname whitelist that recognizes when a request targets a local Ollama or LM Studio instance. When the guard detects destinations such as localhost:11434 or other loopback addresses, it marks the connection as trusted and bypasses rate‑limit quotas that would normally apply to upstream cloud providers.

This means you can run high‑frequency inference jobs against your local GPU without triggering the gateway’s standard throttling mechanisms.

Model Identifier Normalization

The normalizeOllamaModel() function in server/src/routes/ollama.ts sanitizes incoming model strings. For example, a client request specifying llama2:latest is stripped of its tag and mapped to the base identifier expected by the local runtime. This normalization ensures compatibility across client tools that may use different naming conventions for the same underlying weights.

Configuring OpenAI-Compatible Routing

The server/src/providers/openai-compat.ts file contains the translation logic that powers the integration. When FreeLLMAPI receives a request at the /v1/* prefix, it:

1. Rewrites the baseURL to point at your local server (e.g., http://localhost:11434/v1).
2. Translates response fields, extracting message.reasoning from Ollama outputs when present.
3. Forwards streaming chunks with the correct SSE headers for OpenAI SDK compatibility.

This provider also exposes the full Ollama wire format under /api/*, implementing endpoints for tags, chat, generate, show, version, embed, and legacy embeddings as documented in server/src/docs/openapi.ts.

Enabling Ollama Emulation Mode

You can activate the emulation layer through the command line or the web dashboard.

Option 1: Open-loopback mode (no authentication)

This configuration allows any client on your local machine to connect without presenting an API key:

freellmapi config set ollamaEmulation open-loopback

Option 2: Key-required mode (authenticated)

For shared development environments, generate a token and require it in the URL path:

# Create a dedicated token

freellmapi token create --name ollama-token

# Use the token in your requests

curl http://localhost:3001/v1/t/<TOKEN>/chat \
  -H "Content-Type: application/json" \
  -d '{"model":"llama2","messages":[{"role":"user","content":"Hello"}]}'

Connecting Client Applications

Using the Ollama Native API

Send requests directly to the Ollama‑compatible endpoints exposed by FreeLLMAPI:

curl -X POST http://localhost:3001/api/chat \
  -H "Content-Type: application/json" \
  -d '{
        "model": "llama2:latest",
        "messages": [{"role":"user","content":"Explain quantum computing"}]
      }'

Using the OpenAI SDK with LM Studio

Any OpenAI‑compatible SDK can target your local LLM through FreeLLMAPI by changing the baseURL:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:3001/v1",
  apiKey: "dummy",  // Ignored in open-loopback mode
});

const response = await client.chat.completions.create({
  model: "llama2:latest",
  messages: [{ role: "user", content: "Explain quantum tunneling." }],
});

console.log(response.choices[0].message.content);

This approach works identically for LM Studio, which exposes an OpenAI‑compatible server on its own port; FreeLLMAPI simply proxies the traffic while adding its own authentication and logging layers.

Supported Local LLM Runtimes

FreeLLMAPI’s emulation layer is runtime‑agnostic and officially supports:

• Ollama – Primary target with full tag and embedding support.
• LM Studio – OpenAI‑compatible local server mode.
• llama.cpp – When running with -lv or similar OpenAI‑compatible server flags.
• vLLM – Via its built‑in OpenAI‑compatible API implementation.

Any client that understands the Ollama protocol—including the Ollama CLI, Zed editor, and JetBrains AI Assistant—can connect without code changes once the emulation is enabled.

Summary

• FreeLLMAPI connects to local LLMs through an opt‑in Ollama emulation layer that must be enabled via CLI or dashboard.
• The system uses server/src/lib/url-guard.ts to detect local endpoints and exempt them from quotas.
• Model names are normalized via normalizeOllamaModel() in server/src/routes/ollama.ts to handle Ollama‑specific tags.
• The server/src/providers/openai-compat.ts file handles protocol translation between OpenAI‑style and Ollama‑native formats.
• You can run in open-loopback mode for private use or key‑required mode to secure local endpoints with URL‑based tokens.
• Supported runtimes include Ollama, LM Studio, llama.cpp, and vLLM.

Frequently Asked Questions

Does FreeLLMAPI work specifically with LM Studio?

Yes. LM Studio exposes an OpenAI‑compatible local server that FreeLLMAPI can proxy through the same openai-compat provider used for Ollama. Simply point FreeLLMAPI at LM Studio’s local port (typically http://localhost:1234/v1), and the gateway will handle request forwarding and response normalization.

Do I need an API key to connect to local models?

Not in open-loopback mode. This configuration trusts all localhost traffic and ignores the Authorization header. If you enable key‑required mode, you must generate a token via freellmapi token create and include it in the URL path (/v1/t/{token}/...), but the key is only validated by FreeLLMAPI, not passed to the local LLM runtime.

Can I use the standard Ollama CLI with FreeLLMAPI?

Yes. Configure the Ollama CLI to point at FreeLLMAPI’s base URL:

export OLLAMA_HOST=http://localhost:3001
ollama run llama2

The CLI will communicate using the native Ollama protocol, and FreeLLMAPI will route requests to your actual local instance while applying any configured middleware.

Will local requests count against my rate limits?

No. The url-guard.ts module explicitly identifies local hostnames and loopback addresses, marking them as trusted. This exempts the traffic from the quota enforcement and billing logic that applies to cloud‑based providers, allowing unlimited local inference.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}